python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1837233 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 20>10 please execute split_portfolio.py -i 21247982 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2656120'] with mtr_portfolio_ids : ['21247982'] and first list_photo_ids : [] new path : /proc/1837233/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.6167287826538086 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Mar 11 00:54:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3955 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-11 00:54:34.735186: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-11 00:54:34.771125: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-11 00:54:34.774105: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff944000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-11 00:54:34.774159: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-11 00:54:34.780418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-11 00:54:35.104063: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x180405f0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-11 00:54:35.104119: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-11 00:54:35.105486: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 00:54:35.107531: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 00:54:35.114938: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 00:54:35.143755: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 00:54:35.145540: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 00:54:35.187000: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 00:54:35.193544: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 00:54:35.268059: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 00:54:35.269726: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 00:54:35.270391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 00:54:35.271750: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 00:54:35.271782: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 00:54:35.271800: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 00:54:35.273614: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3503 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-11 00:54:35.688842: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 00:54:35.689020: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 00:54:35.689069: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 00:54:35.689108: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 00:54:35.689146: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 00:54:35.689179: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 00:54:35.689216: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 00:54:35.689255: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 00:54:35.690669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 00:54:35.692086: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 00:54:35.692173: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 00:54:35.692206: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 00:54:35.692237: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 00:54:35.692266: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 00:54:35.692296: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 00:54:35.692325: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 00:54:35.692351: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 00:54:35.693482: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 00:54:35.693540: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 00:54:35.693553: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 00:54:35.693566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 00:54:35.694779: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3503 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-11 00:54:49.931491: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 00:54:50.289617: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 00:54:52.425078: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.172969: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-03-11 00:54:53.238185: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.238251: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.239510: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.239545: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.247049: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.247089: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.253572: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.253605: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.254270: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.254292: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.290145: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.290233: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 19.91MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.290252: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-03-11 00:54:53.291421: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.291457: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.292617: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.292652: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.299875: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.299909: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:54:53.300719: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.301528: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.302316: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.310683: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.311366: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.338318: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.339199: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.340042: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.340844: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.345337: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.346198: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.347030: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.347876: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.349730: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.359189: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.359871: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.369022: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.369676: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.370349: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.370987: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.371638: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:54:53.372291: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.92G (3138256896 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 2 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.77500 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 15 Detection mask done ! Trying to reset tf kernel 1837758 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2762 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3955 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0005488395690917969 nb_pixel_total : 17029 time to create 1 rle with old method : 0.019579410552978516 length of segment : 190 time for calcul the mask position with numpy : 0.0006153583526611328 nb_pixel_total : 26474 time to create 1 rle with old method : 0.033966779708862305 length of segment : 180 time for calcul the mask position with numpy : 0.0010955333709716797 nb_pixel_total : 53363 time to create 1 rle with old method : 0.08046960830688477 length of segment : 292 time for calcul the mask position with numpy : 0.0016226768493652344 nb_pixel_total : 60391 time to create 1 rle with old method : 0.11021971702575684 length of segment : 321 time for calcul the mask position with numpy : 0.0009753704071044922 nb_pixel_total : 34961 time to create 1 rle with old method : 0.040288448333740234 length of segment : 218 time for calcul the mask position with numpy : 0.002861499786376953 nb_pixel_total : 121772 time to create 1 rle with old method : 0.13573551177978516 length of segment : 570 time for calcul the mask position with numpy : 0.0005528926849365234 nb_pixel_total : 21290 time to create 1 rle with old method : 0.024999618530273438 length of segment : 201 time for calcul the mask position with numpy : 0.0008790493011474609 nb_pixel_total : 21013 time to create 1 rle with old method : 0.024754047393798828 length of segment : 291 time for calcul the mask position with numpy : 0.0008137226104736328 nb_pixel_total : 34451 time to create 1 rle with old method : 0.04251241683959961 length of segment : 231 time for calcul the mask position with numpy : 0.0026116371154785156 nb_pixel_total : 143596 time to create 1 rle with old method : 0.16512107849121094 length of segment : 371 time for calcul the mask position with numpy : 0.0012295246124267578 nb_pixel_total : 51125 time to create 1 rle with old method : 0.05873274803161621 length of segment : 292 time for calcul the mask position with numpy : 0.0006933212280273438 nb_pixel_total : 29444 time to create 1 rle with old method : 0.034104347229003906 length of segment : 250 time for calcul the mask position with numpy : 0.0018415451049804688 nb_pixel_total : 81114 time to create 1 rle with old method : 0.09433889389038086 length of segment : 307 time for calcul the mask position with numpy : 0.0005733966827392578 nb_pixel_total : 17425 time to create 1 rle with old method : 0.020593643188476562 length of segment : 223 time for calcul the mask position with numpy : 0.00031828880310058594 nb_pixel_total : 9606 time to create 1 rle with old method : 0.01311945915222168 length of segment : 116 time for calcul the mask position with numpy : 9.512901306152344e-05 nb_pixel_total : 745 time to create 1 rle with old method : 0.0010204315185546875 length of segment : 39 time for calcul the mask position with numpy : 0.0019230842590332031 nb_pixel_total : 71234 time to create 1 rle with old method : 0.08124303817749023 length of segment : 410 time for calcul the mask position with numpy : 0.00023651123046875 nb_pixel_total : 2154 time to create 1 rle with old method : 0.002635478973388672 length of segment : 119 time for calcul the mask position with numpy : 0.0022063255310058594 nb_pixel_total : 125297 time to create 1 rle with old method : 0.14499139785766602 length of segment : 349 time for calcul the mask position with numpy : 0.005223751068115234 nb_pixel_total : 107381 time to create 1 rle with old method : 0.14029622077941895 length of segment : 595 time for calcul the mask position with numpy : 0.00034236907958984375 nb_pixel_total : 7656 time to create 1 rle with old method : 0.00981283187866211 length of segment : 139 time for calcul the mask position with numpy : 8.0108642578125e-05 nb_pixel_total : 1595 time to create 1 rle with old method : 0.0022733211517333984 length of segment : 49 time for calcul the mask position with numpy : 0.00029850006103515625 nb_pixel_total : 15553 time to create 1 rle with old method : 0.02362656593322754 length of segment : 151 time for calcul the mask position with numpy : 0.000514984130859375 nb_pixel_total : 11281 time to create 1 rle with old method : 0.017083406448364258 length of segment : 134 time for calcul the mask position with numpy : 0.0004317760467529297 nb_pixel_total : 15636 time to create 1 rle with old method : 0.019594669342041016 length of segment : 152 time for calcul the mask position with numpy : 0.0022585391998291016 nb_pixel_total : 131021 time to create 1 rle with old method : 0.14662694931030273 length of segment : 540 time for calcul the mask position with numpy : 0.501854658126831 nb_pixel_total : 3774572 time to create 1 rle with new method : 0.5464403629302979 length of segment : 4112 time for calcul the mask position with numpy : 0.00023984909057617188 nb_pixel_total : 8502 time to create 1 rle with old method : 0.009957551956176758 length of segment : 184 time for calcul the mask position with numpy : 0.003577709197998047 nb_pixel_total : 96992 time to create 1 rle with old method : 0.11110234260559082 length of segment : 530 time for calcul the mask position with numpy : 0.0006053447723388672 nb_pixel_total : 15540 time to create 1 rle with old method : 0.01788926124572754 length of segment : 206 time for calcul the mask position with numpy : 0.00034546852111816406 nb_pixel_total : 7198 time to create 1 rle with old method : 0.008293390274047852 length of segment : 131 time for calcul the mask position with numpy : 0.0008819103240966797 nb_pixel_total : 25487 time to create 1 rle with old method : 0.029891490936279297 length of segment : 208 time for calcul the mask position with numpy : 0.001977682113647461 nb_pixel_total : 61139 time to create 1 rle with old method : 0.07033061981201172 length of segment : 386 time for calcul the mask position with numpy : 0.002070903778076172 nb_pixel_total : 75717 time to create 1 rle with old method : 0.08645439147949219 length of segment : 413 time for calcul the mask position with numpy : 0.008297204971313477 nb_pixel_total : 248189 time to create 1 rle with new method : 0.030762672424316406 length of segment : 404 time for calcul the mask position with numpy : 0.0008187294006347656 nb_pixel_total : 14001 time to create 1 rle with old method : 0.016801118850708008 length of segment : 183 time for calcul the mask position with numpy : 0.003939151763916016 nb_pixel_total : 139663 time to create 1 rle with old method : 0.1838982105255127 length of segment : 363 time for calcul the mask position with numpy : 0.002431631088256836 nb_pixel_total : 71439 time to create 1 rle with old method : 0.09510183334350586 length of segment : 569 time for calcul the mask position with numpy : 0.0032312870025634766 nb_pixel_total : 65177 time to create 1 rle with old method : 0.07441067695617676 length of segment : 459 time for calcul the mask position with numpy : 0.0010406970977783203 nb_pixel_total : 20411 time to create 1 rle with old method : 0.024301528930664062 length of segment : 248 time for calcul the mask position with numpy : 0.0002493858337402344 nb_pixel_total : 8183 time to create 1 rle with old method : 0.011859893798828125 length of segment : 109 time for calcul the mask position with numpy : 0.0068950653076171875 nb_pixel_total : 202915 time to create 1 rle with new method : 0.010354995727539062 length of segment : 717 time for calcul the mask position with numpy : 0.0013055801391601562 nb_pixel_total : 50688 time to create 1 rle with old method : 0.06212139129638672 length of segment : 466 time for calcul the mask position with numpy : 0.010931968688964844 nb_pixel_total : 316197 time to create 1 rle with new method : 0.036260128021240234 length of segment : 1016 time for calcul the mask position with numpy : 0.00037169456481933594 nb_pixel_total : 3435 time to create 1 rle with old method : 0.004388570785522461 length of segment : 143 time for calcul the mask position with numpy : 0.0007467269897460938 nb_pixel_total : 27088 time to create 1 rle with old method : 0.03730058670043945 length of segment : 171 time for calcul the mask position with numpy : 0.0008342266082763672 nb_pixel_total : 20226 time to create 1 rle with old method : 0.023395061492919922 length of segment : 283 time for calcul the mask position with numpy : 0.00033664703369140625 nb_pixel_total : 9517 time to create 1 rle with old method : 0.011783838272094727 length of segment : 89 time for calcul the mask position with numpy : 0.0008494853973388672 nb_pixel_total : 21058 time to create 1 rle with old method : 0.02593231201171875 length of segment : 231 time for calcul the mask position with numpy : 0.0003185272216796875 nb_pixel_total : 5550 time to create 1 rle with old method : 0.0070040225982666016 length of segment : 138 time for calcul the mask position with numpy : 0.002317190170288086 nb_pixel_total : 76053 time to create 1 rle with old method : 0.09319138526916504 length of segment : 486 time for calcul the mask position with numpy : 0.0005853176116943359 nb_pixel_total : 6390 time to create 1 rle with old method : 0.008718729019165039 length of segment : 122 time for calcul the mask position with numpy : 0.0024771690368652344 nb_pixel_total : 107606 time to create 1 rle with old method : 0.13023018836975098 length of segment : 388 time for calcul the mask position with numpy : 0.00020313262939453125 nb_pixel_total : 1129 time to create 1 rle with old method : 0.0014939308166503906 length of segment : 55 time for calcul the mask position with numpy : 0.0003070831298828125 nb_pixel_total : 5435 time to create 1 rle with old method : 0.006566762924194336 length of segment : 144 time for calcul the mask position with numpy : 0.001544952392578125 nb_pixel_total : 109648 time to create 1 rle with old method : 0.1383368968963623 length of segment : 436 time for calcul the mask position with numpy : 0.008494853973388672 nb_pixel_total : 379463 time to create 1 rle with new method : 0.01358652114868164 length of segment : 701 time for calcul the mask position with numpy : 0.0005829334259033203 nb_pixel_total : 14265 time to create 1 rle with old method : 0.01696300506591797 length of segment : 143 time for calcul the mask position with numpy : 0.004952430725097656 nb_pixel_total : 129849 time to create 1 rle with old method : 0.14715361595153809 length of segment : 754 time for calcul the mask position with numpy : 0.0011255741119384766 nb_pixel_total : 37437 time to create 1 rle with old method : 0.04390311241149902 length of segment : 259 time for calcul the mask position with numpy : 0.0006175041198730469 nb_pixel_total : 19767 time to create 1 rle with old method : 0.023405075073242188 length of segment : 166 time for calcul the mask position with numpy : 0.0003790855407714844 nb_pixel_total : 15012 time to create 1 rle with old method : 0.017489910125732422 length of segment : 122 time for calcul the mask position with numpy : 0.0002090930938720703 nb_pixel_total : 3039 time to create 1 rle with old method : 0.003773927688598633 length of segment : 61 time for calcul the mask position with numpy : 0.001111745834350586 nb_pixel_total : 32320 time to create 1 rle with old method : 0.03692770004272461 length of segment : 271 time for calcul the mask position with numpy : 0.0003311634063720703 nb_pixel_total : 7317 time to create 1 rle with old method : 0.008662939071655273 length of segment : 73 time for calcul the mask position with numpy : 0.0017549991607666016 nb_pixel_total : 37584 time to create 1 rle with old method : 0.044989824295043945 length of segment : 525 time for calcul the mask position with numpy : 0.0037622451782226562 nb_pixel_total : 153359 time to create 1 rle with new method : 0.005656003952026367 length of segment : 637 time for calcul the mask position with numpy : 0.00043129920959472656 nb_pixel_total : 7798 time to create 1 rle with old method : 0.009276390075683594 length of segment : 162 time for calcul the mask position with numpy : 0.0004544258117675781 nb_pixel_total : 6055 time to create 1 rle with old method : 0.00707697868347168 length of segment : 136 time for calcul the mask position with numpy : 0.0003478527069091797 nb_pixel_total : 6304 time to create 1 rle with old method : 0.007602214813232422 length of segment : 147 time for calcul the mask position with numpy : 0.0033397674560546875 nb_pixel_total : 123483 time to create 1 rle with old method : 0.13981103897094727 length of segment : 860 time for calcul the mask position with numpy : 0.00018167495727539062 nb_pixel_total : 8888 time to create 1 rle with old method : 0.010419607162475586 length of segment : 131 time for calcul the mask position with numpy : 0.0005478858947753906 nb_pixel_total : 14969 time to create 1 rle with old method : 0.01777362823486328 length of segment : 114 time for calcul the mask position with numpy : 0.0015468597412109375 nb_pixel_total : 44949 time to create 1 rle with old method : 0.05021214485168457 length of segment : 411 time for calcul the mask position with numpy : 0.000308990478515625 nb_pixel_total : 7926 time to create 1 rle with old method : 0.00942230224609375 length of segment : 103 time for calcul the mask position with numpy : 0.00016164779663085938 nb_pixel_total : 2843 time to create 1 rle with old method : 0.005166053771972656 length of segment : 51 time for calcul the mask position with numpy : 0.0002346038818359375 nb_pixel_total : 7301 time to create 1 rle with old method : 0.008824348449707031 length of segment : 62 time for calcul the mask position with numpy : 0.0016598701477050781 nb_pixel_total : 67983 time to create 1 rle with old method : 0.07939600944519043 length of segment : 401 time for calcul the mask position with numpy : 0.0010192394256591797 nb_pixel_total : 40632 time to create 1 rle with old method : 0.04596900939941406 length of segment : 230 time for calcul the mask position with numpy : 0.0001552104949951172 nb_pixel_total : 1491 time to create 1 rle with old method : 0.0019330978393554688 length of segment : 39 time for calcul the mask position with numpy : 0.0012614727020263672 nb_pixel_total : 58514 time to create 1 rle with old method : 0.06655406951904297 length of segment : 297 time for calcul the mask position with numpy : 0.0010814666748046875 nb_pixel_total : 62042 time to create 1 rle with old method : 0.07288169860839844 length of segment : 364 time for calcul the mask position with numpy : 0.0005650520324707031 nb_pixel_total : 16125 time to create 1 rle with old method : 0.01958298683166504 length of segment : 226 time for calcul the mask position with numpy : 0.0007698535919189453 nb_pixel_total : 34058 time to create 1 rle with old method : 0.0382843017578125 length of segment : 171 time for calcul the mask position with numpy : 0.0017199516296386719 nb_pixel_total : 73544 time to create 1 rle with old method : 0.08290529251098633 length of segment : 302 time for calcul the mask position with numpy : 0.00014591217041015625 nb_pixel_total : 1366 time to create 1 rle with old method : 0.0017910003662109375 length of segment : 42 time for calcul the mask position with numpy : 0.0011966228485107422 nb_pixel_total : 62521 time to create 1 rle with old method : 0.07028889656066895 length of segment : 291 time for calcul the mask position with numpy : 0.00136566162109375 nb_pixel_total : 66564 time to create 1 rle with old method : 0.0927889347076416 length of segment : 255 time for calcul the mask position with numpy : 0.0003955364227294922 nb_pixel_total : 10308 time to create 1 rle with old method : 0.011857748031616211 length of segment : 146 time for calcul the mask position with numpy : 0.0011515617370605469 nb_pixel_total : 67590 time to create 1 rle with old method : 0.0765981674194336 length of segment : 252 time for calcul the mask position with numpy : 0.0008969306945800781 nb_pixel_total : 28018 time to create 1 rle with old method : 0.04529857635498047 length of segment : 307 time for calcul the mask position with numpy : 0.0068340301513671875 nb_pixel_total : 338339 time to create 1 rle with new method : 0.012392044067382812 length of segment : 677 time for calcul the mask position with numpy : 0.00024080276489257812 nb_pixel_total : 3619 time to create 1 rle with old method : 0.004361629486083984 length of segment : 133 time for calcul the mask position with numpy : 0.001352071762084961 nb_pixel_total : 33986 time to create 1 rle with old method : 0.03846931457519531 length of segment : 276 time for calcul the mask position with numpy : 0.0027053356170654297 nb_pixel_total : 61409 time to create 1 rle with old method : 0.0755312442779541 length of segment : 396 time for calcul the mask position with numpy : 0.0009768009185791016 nb_pixel_total : 26991 time to create 1 rle with old method : 0.03363943099975586 length of segment : 208 time for calcul the mask position with numpy : 0.01917099952697754 nb_pixel_total : 447105 time to create 1 rle with new method : 0.06999540328979492 length of segment : 1091 time for calcul the mask position with numpy : 0.0002512931823730469 nb_pixel_total : 3010 time to create 1 rle with old method : 0.0037965774536132812 length of segment : 51 time for calcul the mask position with numpy : 0.0008919239044189453 nb_pixel_total : 28652 time to create 1 rle with old method : 0.037744998931884766 length of segment : 258 time for calcul the mask position with numpy : 0.0055348873138427734 nb_pixel_total : 118168 time to create 1 rle with old method : 0.16265511512756348 length of segment : 539 time for calcul the mask position with numpy : 0.0009405612945556641 nb_pixel_total : 16077 time to create 1 rle with old method : 0.01882767677307129 length of segment : 194 time for calcul the mask position with numpy : 0.00010204315185546875 nb_pixel_total : 2379 time to create 1 rle with old method : 0.002930879592895508 length of segment : 61 time for calcul the mask position with numpy : 0.003438711166381836 nb_pixel_total : 103567 time to create 1 rle with old method : 0.11739778518676758 length of segment : 325 time for calcul the mask position with numpy : 0.003958463668823242 nb_pixel_total : 37885 time to create 1 rle with old method : 0.04353213310241699 length of segment : 517 time for calcul the mask position with numpy : 0.0013854503631591797 nb_pixel_total : 13388 time to create 1 rle with old method : 0.016244888305664062 length of segment : 252 time for calcul the mask position with numpy : 0.005251407623291016 nb_pixel_total : 149809 time to create 1 rle with old method : 0.16612529754638672 length of segment : 558 time for calcul the mask position with numpy : 0.0011448860168457031 nb_pixel_total : 20369 time to create 1 rle with old method : 0.027822017669677734 length of segment : 190 time for calcul the mask position with numpy : 0.00015783309936523438 nb_pixel_total : 3421 time to create 1 rle with old method : 0.004111766815185547 length of segment : 101 time for calcul the mask position with numpy : 0.002452850341796875 nb_pixel_total : 61765 time to create 1 rle with old method : 0.07011008262634277 length of segment : 315 time for calcul the mask position with numpy : 0.0023093223571777344 nb_pixel_total : 57654 time to create 1 rle with old method : 0.0657510757446289 length of segment : 402 time for calcul the mask position with numpy : 0.0024781227111816406 nb_pixel_total : 35183 time to create 1 rle with old method : 0.03995656967163086 length of segment : 476 time for calcul the mask position with numpy : 0.0026369094848632812 nb_pixel_total : 61862 time to create 1 rle with old method : 0.06856346130371094 length of segment : 417 time for calcul the mask position with numpy : 0.01222991943359375 nb_pixel_total : 365389 time to create 1 rle with new method : 0.025352954864501953 length of segment : 486 time for calcul the mask position with numpy : 0.0006754398345947266 nb_pixel_total : 27270 time to create 1 rle with old method : 0.0357060432434082 length of segment : 310 time for calcul the mask position with numpy : 0.0011489391326904297 nb_pixel_total : 56121 time to create 1 rle with old method : 0.06445074081420898 length of segment : 397 time for calcul the mask position with numpy : 0.001224517822265625 nb_pixel_total : 56852 time to create 1 rle with old method : 0.06804943084716797 length of segment : 286 time for calcul the mask position with numpy : 0.004290342330932617 nb_pixel_total : 294891 time to create 1 rle with new method : 0.010200738906860352 length of segment : 394 time for calcul the mask position with numpy : 0.001520395278930664 nb_pixel_total : 47881 time to create 1 rle with old method : 0.054543495178222656 length of segment : 245 time for calcul the mask position with numpy : 0.0013043880462646484 nb_pixel_total : 37235 time to create 1 rle with old method : 0.055190324783325195 length of segment : 484 time for calcul the mask position with numpy : 0.001973867416381836 nb_pixel_total : 95982 time to create 1 rle with old method : 0.11121201515197754 length of segment : 424 time for calcul the mask position with numpy : 0.0010437965393066406 nb_pixel_total : 40131 time to create 1 rle with old method : 0.0462946891784668 length of segment : 264 time for calcul the mask position with numpy : 0.001867055892944336 nb_pixel_total : 67132 time to create 1 rle with old method : 0.07741117477416992 length of segment : 436 time for calcul the mask position with numpy : 0.00023102760314941406 nb_pixel_total : 3361 time to create 1 rle with old method : 0.0041065216064453125 length of segment : 82 time for calcul the mask position with numpy : 0.0006120204925537109 nb_pixel_total : 19111 time to create 1 rle with old method : 0.022379398345947266 length of segment : 208 time for calcul the mask position with numpy : 0.004389286041259766 nb_pixel_total : 153747 time to create 1 rle with new method : 0.007344484329223633 length of segment : 583 time for calcul the mask position with numpy : 0.0013804435729980469 nb_pixel_total : 57254 time to create 1 rle with old method : 0.06434822082519531 length of segment : 388 time for calcul the mask position with numpy : 0.00023603439331054688 nb_pixel_total : 1778 time to create 1 rle with old method : 0.0023298263549804688 length of segment : 60 time for calcul the mask position with numpy : 0.0005528926849365234 nb_pixel_total : 29368 time to create 1 rle with old method : 0.033971548080444336 length of segment : 241 time for calcul the mask position with numpy : 0.0018918514251708984 nb_pixel_total : 59635 time to create 1 rle with old method : 0.07164883613586426 length of segment : 359 time for calcul the mask position with numpy : 0.0022852420806884766 nb_pixel_total : 67697 time to create 1 rle with old method : 0.07509207725524902 length of segment : 293 time for calcul the mask position with numpy : 0.0008292198181152344 nb_pixel_total : 21326 time to create 1 rle with old method : 0.024966001510620117 length of segment : 440 time for calcul the mask position with numpy : 0.0006046295166015625 nb_pixel_total : 14825 time to create 1 rle with old method : 0.01790904998779297 length of segment : 148 time for calcul the mask position with numpy : 0.00027871131896972656 nb_pixel_total : 1206 time to create 1 rle with old method : 0.0014615058898925781 length of segment : 139 time for calcul the mask position with numpy : 0.00042319297790527344 nb_pixel_total : 10651 time to create 1 rle with old method : 0.012786149978637695 length of segment : 137 time for calcul the mask position with numpy : 0.0032520294189453125 nb_pixel_total : 122290 time to create 1 rle with old method : 0.1366863250732422 length of segment : 387 time for calcul the mask position with numpy : 0.0015106201171875 nb_pixel_total : 63655 time to create 1 rle with old method : 0.07157254219055176 length of segment : 291 time for calcul the mask position with numpy : 0.00563812255859375 nb_pixel_total : 118735 time to create 1 rle with old method : 0.13230133056640625 length of segment : 453 time for calcul the mask position with numpy : 0.0022106170654296875 nb_pixel_total : 124053 time to create 1 rle with old method : 0.14441251754760742 length of segment : 553 time for calcul the mask position with numpy : 0.0017521381378173828 nb_pixel_total : 110413 time to create 1 rle with old method : 0.12341904640197754 length of segment : 327 time for calcul the mask position with numpy : 0.0008175373077392578 nb_pixel_total : 18753 time to create 1 rle with old method : 0.02129507064819336 length of segment : 249 time for calcul the mask position with numpy : 0.007210493087768555 nb_pixel_total : 354679 time to create 1 rle with new method : 0.011425018310546875 length of segment : 752 time for calcul the mask position with numpy : 0.00028133392333984375 nb_pixel_total : 7442 time to create 1 rle with old method : 0.014611959457397461 length of segment : 96 time for calcul the mask position with numpy : 0.00032401084899902344 nb_pixel_total : 7162 time to create 1 rle with old method : 0.008827447891235352 length of segment : 166 time spent for convertir_results : 16.593791961669922 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 149 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 46847 save missing photos in datou_result : time spend for datou_step_exec : 81.47105431556702 time spend to save output : 25.581000804901123 total time spend for step 1 : 107.05205512046814 step2:crop_condition Tue Mar 11 00:56:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 149 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 38 About to insert : list_path_to_insert length 38 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1342515556, 1342515358, 1342515354, 1342515351, 1342515347, 1342515343, 1342515340, 1342515147, 1342515144, 1342515140] Looping around the photos to save general results len do output : 94 /-3702943349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943351Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943352Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703832723Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703832722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703832720Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515556', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515358', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515354', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515351', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515347', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515343', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515147', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515144', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515140', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 292 time used for this insertion : 0.03584575653076172 save_final save missing photos in datou_result : time spend for datou_step_exec : 38.67694044113159 time spend to save output : 0.03947162628173828 total time spend for step 2 : 38.71641206741333 step3:thcl Tue Mar 11 00:56:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.009833574295043945 time to convert the images to numpy array : 0.11935925483703613 time to import caffe and check if the image exist : 0.013271808624267578 time to convert the images to numpy array : 0.1552426815032959 time to import caffe and check if the image exist : 0.01232767105102539 time to convert the images to numpy array : 0.16133928298950195 time to import caffe and check if the image exist : 0.013616323471069336 time to convert the images to numpy array : 0.16831684112548828 time to import caffe and check if the image exist : 0.013410091400146484 time to convert the images to numpy array : 0.20261573791503906 time to import caffe and check if the image exist : 0.014721393585205078 time to convert the images to numpy array : 0.21138596534729004 time to import caffe and check if the image exist : 0.01694965362548828 time to convert the images to numpy array : 0.23347926139831543 time to import caffe and check if the image exist : 0.01852250099182129 time to convert the images to numpy array : 0.2460012435913086 time to import caffe and check if the image exist : 0.014649629592895508 time to convert the images to numpy array : 0.2741403579711914 time to import caffe and check if the image exist : 0.01714181900024414 time to convert the images to numpy array : 0.2756984233856201 total time to convert the images to numpy array : 0.6599187850952148 list photo_ids error: [] list photo_ids correct : [-3702943465, -3702943487, -3702943397, -3702943438, -3702943362, -3702943361, -3702943389, -3702943426, -3702943436, -3702943444, -3702943482, -3702943353, -3702943412, -3702943413, -3702943349, -3702943356, -3702943351, -3702943384, -3702943387, -3702943377, -3702943380, -3702943381, -3702943396, -3702943431, -3702943435, -3702943432, -3702943434, -3702943446, -3702943472, -3702943352, -3702943391, -3702943385, -3702943401, -3702943399, -3702943427, -3702943437, -3702943433, -3702943439, -3702943460, -3702943447, -3702943467, -3702943464, -3702943454, -3702943455, -3702943451, -3702943476, -3702943374, -3702943359, -3702943358, -3702943350, -3702943371, -3702943354, -3702943366, -3702943395, -3702943463, -3702943452, -3702943477, -3702943470, -3702943473, -3702943478, -3702943468, -3702943469, -3702943486, -3702943483, -3702943388, -3703832720, -3702943404, -3702943407, -3702943419, -3702943414, -3702943422, -3702943408, -3702943410, -3702943415, -3702943466, -3702943443, -3702943462, -3702943449, -3702943474, -3702943479, -3702943485, -3703832723, -3702943367, -3702943360, -3702943373, -3702943357, -3702943364, -3703832722, -3702943405, -3702943420, -3702943429, -3702943440, -3702943458, -3702943457] number of photos to traite : 94 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2541 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 2322 wait 20 seconds l 3637 free memory gpu now : 2322 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.020233154296875 time used to do the prediction : 0.30700063705444336 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 2320 wait 20 seconds l 3637 free memory gpu now : 2320 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 2320 wait 20 seconds l 3637 free memory gpu now : 2320 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 5.332836866378784 time used to do the prediction : 0.44580745697021484 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 0.0017633438110351562 save missing photos in datou_result : time spend for datou_step_exec : 75.28959584236145 time spend to save output : 0.010429620742797852 total time spend for step 3 : 75.30002546310425 step4:merge_mask_thcl_custom Tue Mar 11 00:58:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 149 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1342515556, 1342515358, 1342515354, 1342515351, 1342515347, 1342515343, 1342515340, 1342515147, 1342515144, 1342515140] Looping around the photos to save general results len do output : 10 /1342515556Didn't retrieve data .Didn't retrieve data . /1342515358Didn't retrieve data .Didn't retrieve data . /1342515354Didn't retrieve data .Didn't retrieve data . /1342515351Didn't retrieve data .Didn't retrieve data . /1342515347Didn't retrieve data .Didn't retrieve data . /1342515343Didn't retrieve data .Didn't retrieve data . /1342515340Didn't retrieve data .Didn't retrieve data . /1342515147Didn't retrieve data .Didn't retrieve data . /1342515144Didn't retrieve data .Didn't retrieve data . /1342515140Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515556', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515358', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515354', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515351', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515347', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515343', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515147', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515144', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515140', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 2.4827864170074463 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.0923759937286377 time spend to save output : 2.4859447479248047 total time spend for step 4 : 2.5783207416534424 step5:rle_unique_nms_with_priority Tue Mar 11 00:58:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 25 nb_hashtags : 8 time to prepare the origin masks : 23.038759231567383 time for calcul the mask position with numpy : 0.816138505935669 nb_pixel_total : 6113921 time to create 1 rle with new method : 1.103654384613037 time for calcul the mask position with numpy : 0.0398252010345459 nb_pixel_total : 262 time to create 1 rle with old method : 0.00042176246643066406 time for calcul the mask position with numpy : 0.03858590126037598 nb_pixel_total : 130919 time to create 1 rle with old method : 0.17108392715454102 time for calcul the mask position with numpy : 0.0373539924621582 nb_pixel_total : 2411 time to create 1 rle with old method : 0.0029175281524658203 time for calcul the mask position with numpy : 0.038095712661743164 nb_pixel_total : 31668 time to create 1 rle with old method : 0.03714108467102051 time for calcul the mask position with numpy : 0.03488039970397949 nb_pixel_total : 21232 time to create 1 rle with old method : 0.023451805114746094 time for calcul the mask position with numpy : 0.038346290588378906 nb_pixel_total : 9555 time to create 1 rle with old method : 0.010731697082519531 time for calcul the mask position with numpy : 0.04043006896972656 nb_pixel_total : 20263 time to create 1 rle with old method : 0.023920059204101562 time for calcul the mask position with numpy : 0.030228614807128906 nb_pixel_total : 53189 time to create 1 rle with old method : 0.061284542083740234 time for calcul the mask position with numpy : 0.022984981536865234 nb_pixel_total : 34824 time to create 1 rle with old method : 0.04003739356994629 time for calcul the mask position with numpy : 0.022179603576660156 nb_pixel_total : 1579 time to create 1 rle with old method : 0.0018143653869628906 time for calcul the mask position with numpy : 0.02269911766052246 nb_pixel_total : 7616 time to create 1 rle with old method : 0.008642911911010742 time for calcul the mask position with numpy : 0.023078203201293945 nb_pixel_total : 10 time to create 1 rle with old method : 5.269050598144531e-05 time for calcul the mask position with numpy : 0.02401280403137207 nb_pixel_total : 29279 time to create 1 rle with old method : 0.03350234031677246 time for calcul the mask position with numpy : 0.0252993106842041 nb_pixel_total : 80533 time to create 1 rle with old method : 0.0894770622253418 time for calcul the mask position with numpy : 0.026022911071777344 nb_pixel_total : 71022 time to create 1 rle with old method : 0.0791475772857666 time for calcul the mask position with numpy : 0.02359175682067871 nb_pixel_total : 2124 time to create 1 rle with old method : 0.0028076171875 time for calcul the mask position with numpy : 0.02522754669189453 nb_pixel_total : 17398 time to create 1 rle with old method : 0.019568204879760742 time for calcul the mask position with numpy : 0.025527000427246094 nb_pixel_total : 738 time to create 1 rle with old method : 0.0008919239044189453 time for calcul the mask position with numpy : 0.027332067489624023 nb_pixel_total : 51076 time to create 1 rle with old method : 0.05624890327453613 time for calcul the mask position with numpy : 0.02311563491821289 nb_pixel_total : 26359 time to create 1 rle with old method : 0.030173540115356445 time for calcul the mask position with numpy : 0.023339509963989258 nb_pixel_total : 20836 time to create 1 rle with old method : 0.026063203811645508 time for calcul the mask position with numpy : 0.023862838745117188 nb_pixel_total : 121489 time to create 1 rle with old method : 0.14020657539367676 time for calcul the mask position with numpy : 0.024745941162109375 nb_pixel_total : 124939 time to create 1 rle with old method : 0.13991498947143555 time for calcul the mask position with numpy : 0.026331424713134766 nb_pixel_total : 60025 time to create 1 rle with old method : 0.06772971153259277 time for calcul the mask position with numpy : 0.025490283966064453 nb_pixel_total : 16973 time to create 1 rle with old method : 0.01933908462524414 create new chi : 3.7634756565093994 time to delete rle : 0.017061233520507812 batch 1 Loaded 27 chid ids of type : 3726 Number RLEs to save : 13929 TO DO : save crop sub photo not yet done ! save time : 4.604490041732788 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 3.2250916957855225 time for calcul the mask position with numpy : 0.5774555206298828 nb_pixel_total : 3111433 time to create 1 rle with new method : 0.9151022434234619 time for calcul the mask position with numpy : 0.02362370491027832 nb_pixel_total : 8489 time to create 1 rle with old method : 0.009928464889526367 time for calcul the mask position with numpy : 0.5858151912689209 nb_pixel_total : 3930318 time to create 1 rle with new method : 1.286672830581665 create new chi : 3.47235107421875 time to delete rle : 0.0009469985961914062 batch 1 Loaded 3 chid ids of type : 3726 Number RLEs to save : 9756 TO DO : save crop sub photo not yet done ! save time : 1.0041096210479736 nb_obj : 21 nb_hashtags : 7 time to prepare the origin masks : 17.911080598831177 time for calcul the mask position with numpy : 0.7271404266357422 nb_pixel_total : 5662625 time to create 1 rle with new method : 0.9288060665130615 time for calcul the mask position with numpy : 0.026473522186279297 nb_pixel_total : 25447 time to create 1 rle with old method : 0.02951359748840332 time for calcul the mask position with numpy : 0.027579784393310547 nb_pixel_total : 6807 time to create 1 rle with old method : 0.011156797409057617 time for calcul the mask position with numpy : 0.029696226119995117 nb_pixel_total : 13940 time to create 1 rle with old method : 0.018758773803710938 time for calcul the mask position with numpy : 0.02600240707397461 nb_pixel_total : 20190 time to create 1 rle with old method : 0.022171735763549805 time for calcul the mask position with numpy : 0.02720475196838379 nb_pixel_total : 3396 time to create 1 rle with old method : 0.004276275634765625 time for calcul the mask position with numpy : 0.024278640747070312 nb_pixel_total : 1574 time to create 1 rle with old method : 0.002194643020629883 time for calcul the mask position with numpy : 0.029550552368164062 nb_pixel_total : 299633 time to create 1 rle with new method : 1.111943006515503 time for calcul the mask position with numpy : 0.026993274688720703 nb_pixel_total : 202494 time to create 1 rle with new method : 0.6525356769561768 time for calcul the mask position with numpy : 0.02821660041809082 nb_pixel_total : 117331 time to create 1 rle with old method : 0.13276386260986328 time for calcul the mask position with numpy : 0.026041746139526367 nb_pixel_total : 8164 time to create 1 rle with old method : 0.009092330932617188 time for calcul the mask position with numpy : 0.02410578727722168 nb_pixel_total : 96715 time to create 1 rle with old method : 0.10918831825256348 time for calcul the mask position with numpy : 0.02657485008239746 nb_pixel_total : 20321 time to create 1 rle with old method : 0.02280592918395996 time for calcul the mask position with numpy : 0.02532029151916504 nb_pixel_total : 27000 time to create 1 rle with old method : 0.0761420726776123 time for calcul the mask position with numpy : 0.026222944259643555 nb_pixel_total : 65010 time to create 1 rle with old method : 0.07230019569396973 time for calcul the mask position with numpy : 0.02670764923095703 nb_pixel_total : 71123 time to create 1 rle with old method : 0.08260416984558105 time for calcul the mask position with numpy : 0.025165081024169922 nb_pixel_total : 1330 time to create 1 rle with old method : 0.0026836395263671875 time for calcul the mask position with numpy : 0.02964949607849121 nb_pixel_total : 247305 time to create 1 rle with new method : 0.8635702133178711 time for calcul the mask position with numpy : 0.02606058120727539 nb_pixel_total : 76178 time to create 1 rle with old method : 0.08637189865112305 time for calcul the mask position with numpy : 0.02491450309753418 nb_pixel_total : 7165 time to create 1 rle with old method : 0.008402109146118164 time for calcul the mask position with numpy : 0.025501012802124023 nb_pixel_total : 61045 time to create 1 rle with old method : 0.06879830360412598 time for calcul the mask position with numpy : 0.023448705673217773 nb_pixel_total : 15447 time to create 1 rle with old method : 0.016757726669311523 create new chi : 5.749244213104248 time to delete rle : 0.0018687248229980469 batch 1 Loaded 25 chid ids of type : 3726 Number RLEs to save : 15888 TO DO : save crop sub photo not yet done ! save time : 9.482601165771484 nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 10.886421203613281 time for calcul the mask position with numpy : 0.8837523460388184 nb_pixel_total : 6320849 time to create 1 rle with new method : 0.9103288650512695 time for calcul the mask position with numpy : 0.03432297706604004 nb_pixel_total : 20011 time to create 1 rle with old method : 0.02147388458251953 time for calcul the mask position with numpy : 0.033702850341796875 nb_pixel_total : 5387 time to create 1 rle with old method : 0.005656719207763672 time for calcul the mask position with numpy : 0.03368496894836426 nb_pixel_total : 1123 time to create 1 rle with old method : 0.001191854476928711 time for calcul the mask position with numpy : 0.03543543815612793 nb_pixel_total : 9410 time to create 1 rle with old method : 0.01030874252319336 time for calcul the mask position with numpy : 0.026176929473876953 nb_pixel_total : 7044 time to create 1 rle with old method : 0.007657289505004883 time for calcul the mask position with numpy : 0.02511739730834961 nb_pixel_total : 379211 time to create 1 rle with new method : 0.7378149032592773 time for calcul the mask position with numpy : 0.022068262100219727 nb_pixel_total : 59837 time to create 1 rle with old method : 0.0655815601348877 time for calcul the mask position with numpy : 0.02355790138244629 nb_pixel_total : 107442 time to create 1 rle with old method : 0.11783456802368164 time for calcul the mask position with numpy : 0.024615764617919922 nb_pixel_total : 4618 time to create 1 rle with old method : 0.006558418273925781 time for calcul the mask position with numpy : 0.023502588272094727 nb_pixel_total : 5529 time to create 1 rle with old method : 0.006169557571411133 time for calcul the mask position with numpy : 0.024399757385253906 nb_pixel_total : 129779 time to create 1 rle with old method : 0.1366257667541504 create new chi : 3.284250259399414 time to delete rle : 0.0009441375732421875 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 8501 TO DO : save crop sub photo not yet done ! save time : 3.6155877113342285 nb_obj : 14 nb_hashtags : 6 time to prepare the origin masks : 8.948337316513062 time for calcul the mask position with numpy : 0.5499155521392822 nb_pixel_total : 6560134 time to create 1 rle with new method : 0.618154764175415 time for calcul the mask position with numpy : 0.021514415740966797 nb_pixel_total : 29795 time to create 1 rle with old method : 0.034399986267089844 time for calcul the mask position with numpy : 0.023427248001098633 nb_pixel_total : 123129 time to create 1 rle with old method : 0.13542771339416504 time for calcul the mask position with numpy : 0.021342992782592773 nb_pixel_total : 7282 time to create 1 rle with old method : 0.007987737655639648 time for calcul the mask position with numpy : 0.021105289459228516 nb_pixel_total : 37344 time to create 1 rle with old method : 0.040993690490722656 time for calcul the mask position with numpy : 0.021541595458984375 nb_pixel_total : 4499 time to create 1 rle with old method : 0.0050313472747802734 time for calcul the mask position with numpy : 0.023375988006591797 nb_pixel_total : 14942 time to create 1 rle with old method : 0.015917539596557617 time for calcul the mask position with numpy : 0.021847963333129883 nb_pixel_total : 6284 time to create 1 rle with old method : 0.007933378219604492 time for calcul the mask position with numpy : 0.02238321304321289 nb_pixel_total : 32252 time to create 1 rle with old method : 0.03480243682861328 time for calcul the mask position with numpy : 0.022949695587158203 nb_pixel_total : 6036 time to create 1 rle with old method : 0.006666898727416992 time for calcul the mask position with numpy : 0.022258996963500977 nb_pixel_total : 7740 time to create 1 rle with old method : 0.008430957794189453 time for calcul the mask position with numpy : 0.023863792419433594 nb_pixel_total : 3022 time to create 1 rle with old method : 0.003444194793701172 time for calcul the mask position with numpy : 0.022907018661499023 nb_pixel_total : 44844 time to create 1 rle with old method : 0.05616188049316406 time for calcul the mask position with numpy : 0.026594161987304688 nb_pixel_total : 153271 time to create 1 rle with new method : 0.5275859832763672 time for calcul the mask position with numpy : 0.02332925796508789 nb_pixel_total : 19666 time to create 1 rle with old method : 0.022095203399658203 create new chi : 2.4541971683502197 time to delete rle : 0.0010304450988769531 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 9785 TO DO : save crop sub photo not yet done ! save time : 8.349987268447876 nb_obj : 6 nb_hashtags : 4 time to prepare the origin masks : 4.4296181201934814 time for calcul the mask position with numpy : 0.2559621334075928 nb_pixel_total : 6871985 time to create 1 rle with new method : 0.5683937072753906 time for calcul the mask position with numpy : 0.024031400680541992 nb_pixel_total : 1488 time to create 1 rle with old method : 0.001718282699584961 time for calcul the mask position with numpy : 0.02098393440246582 nb_pixel_total : 40536 time to create 1 rle with old method : 0.045183420181274414 time for calcul the mask position with numpy : 0.0223233699798584 nb_pixel_total : 67795 time to create 1 rle with old method : 0.07451272010803223 time for calcul the mask position with numpy : 0.023145437240600586 nb_pixel_total : 7251 time to create 1 rle with old method : 0.008256196975708008 time for calcul the mask position with numpy : 0.022586822509765625 nb_pixel_total : 2817 time to create 1 rle with old method : 0.0031898021697998047 time for calcul the mask position with numpy : 0.021465778350830078 nb_pixel_total : 58368 time to create 1 rle with old method : 0.0659632682800293 create new chi : 1.1944169998168945 time to delete rle : 0.0005307197570800781 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 4332 TO DO : save crop sub photo not yet done ! save time : 3.168717861175537 nb_obj : 11 nb_hashtags : 6 time to prepare the origin masks : 13.395166158676147 time for calcul the mask position with numpy : 0.9639582633972168 nb_pixel_total : 6477786 time to create 1 rle with new method : 1.851144790649414 time for calcul the mask position with numpy : 0.037015676498413086 nb_pixel_total : 10263 time to create 1 rle with old method : 0.012044668197631836 time for calcul the mask position with numpy : 0.02869105339050293 nb_pixel_total : 3600 time to create 1 rle with old method : 0.0042188167572021484 time for calcul the mask position with numpy : 0.023012638092041016 nb_pixel_total : 73391 time to create 1 rle with old method : 0.08363485336303711 time for calcul the mask position with numpy : 0.022258281707763672 nb_pixel_total : 27983 time to create 1 rle with old method : 0.03113245964050293 time for calcul the mask position with numpy : 0.0221712589263916 nb_pixel_total : 16068 time to create 1 rle with old method : 0.018465518951416016 time for calcul the mask position with numpy : 0.024012327194213867 nb_pixel_total : 1476 time to create 1 rle with old method : 0.002051115036010742 time for calcul the mask position with numpy : 0.024422407150268555 nb_pixel_total : 5424 time to create 1 rle with old method : 0.006438732147216797 time for calcul the mask position with numpy : 0.024854183197021484 nb_pixel_total : 61333 time to create 1 rle with old method : 0.06921267509460449 time for calcul the mask position with numpy : 0.021845579147338867 nb_pixel_total : 33917 time to create 1 rle with old method : 0.04304027557373047 time for calcul the mask position with numpy : 0.024440526962280273 nb_pixel_total : 337643 time to create 1 rle with new method : 1.1147608757019043 time for calcul the mask position with numpy : 0.025485992431640625 nb_pixel_total : 1356 time to create 1 rle with old method : 0.002408266067504883 create new chi : 4.559538841247559 time to delete rle : 0.0012099742889404297 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 7417 TO DO : save crop sub photo not yet done ! save time : 5.419132471084595 nb_obj : 25 nb_hashtags : 6 time to prepare the origin masks : 19.366838693618774 time for calcul the mask position with numpy : 0.475757360458374 nb_pixel_total : 5375581 time to create 1 rle with new method : 0.4498751163482666 time for calcul the mask position with numpy : 0.033669233322143555 nb_pixel_total : 15491 time to create 1 rle with old method : 0.018221139907836914 time for calcul the mask position with numpy : 0.0288088321685791 nb_pixel_total : 134 time to create 1 rle with old method : 0.000530242919921875 time for calcul the mask position with numpy : 0.028166770935058594 nb_pixel_total : 1916 time to create 1 rle with old method : 0.0030279159545898438 time for calcul the mask position with numpy : 0.03014540672302246 nb_pixel_total : 61628 time to create 1 rle with old method : 0.07484078407287598 time for calcul the mask position with numpy : 0.02685379981994629 nb_pixel_total : 20695 time to create 1 rle with old method : 0.028981685638427734 time for calcul the mask position with numpy : 0.029743671417236328 nb_pixel_total : 61544 time to create 1 rle with old method : 0.09769320487976074 time for calcul the mask position with numpy : 0.03486943244934082 nb_pixel_total : 355374 time to create 1 rle with new method : 0.4295501708984375 time for calcul the mask position with numpy : 0.027797460556030273 nb_pixel_total : 2665 time to create 1 rle with old method : 0.0030252933502197266 time for calcul the mask position with numpy : 0.029514551162719727 nb_pixel_total : 26940 time to create 1 rle with old method : 0.031438589096069336 time for calcul the mask position with numpy : 0.02749037742614746 nb_pixel_total : 13376 time to create 1 rle with old method : 0.015237092971801758 time for calcul the mask position with numpy : 0.02792811393737793 nb_pixel_total : 2743 time to create 1 rle with old method : 0.003834247589111328 time for calcul the mask position with numpy : 0.03079080581665039 nb_pixel_total : 37203 time to create 1 rle with old method : 0.044564247131347656 time for calcul the mask position with numpy : 0.028925180435180664 nb_pixel_total : 47767 time to create 1 rle with old method : 0.0551600456237793 time for calcul the mask position with numpy : 0.02736043930053711 nb_pixel_total : 28584 time to create 1 rle with old method : 0.03383660316467285 time for calcul the mask position with numpy : 0.02954697608947754 nb_pixel_total : 2355 time to create 1 rle with old method : 0.0038182735443115234 time for calcul the mask position with numpy : 0.0319056510925293 nb_pixel_total : 117890 time to create 1 rle with old method : 0.14023303985595703 time for calcul the mask position with numpy : 0.03106832504272461 nb_pixel_total : 1879 time to create 1 rle with old method : 0.002500295639038086 time for calcul the mask position with numpy : 0.03213143348693848 nb_pixel_total : 445896 time to create 1 rle with new method : 0.3872060775756836 time for calcul the mask position with numpy : 0.0260772705078125 nb_pixel_total : 149873 time to create 1 rle with old method : 0.17171573638916016 time for calcul the mask position with numpy : 0.02615070343017578 nb_pixel_total : 56009 time to create 1 rle with old method : 0.06218767166137695 time for calcul the mask position with numpy : 0.029445886611938477 nb_pixel_total : 37548 time to create 1 rle with old method : 0.042362213134765625 time for calcul the mask position with numpy : 0.027619600296020508 nb_pixel_total : 33918 time to create 1 rle with old method : 0.038992881774902344 time for calcul the mask position with numpy : 0.027032852172851562 nb_pixel_total : 20265 time to create 1 rle with old method : 0.02380514144897461 time for calcul the mask position with numpy : 0.025953054428100586 nb_pixel_total : 29634 time to create 1 rle with old method : 0.03694581985473633 time for calcul the mask position with numpy : 0.03312182426452637 nb_pixel_total : 103332 time to create 1 rle with old method : 0.14203619956970215 create new chi : 3.6504769325256348 time to delete rle : 0.0025589466094970703 batch 1 Loaded 26 chid ids of type : 3726 Number RLEs to save : 17536 TO DO : save crop sub photo not yet done ! save time : 5.327585935592651 nb_obj : 12 nb_hashtags : 6 time to prepare the origin masks : 11.263652801513672 time for calcul the mask position with numpy : 0.6716418266296387 nb_pixel_total : 6508770 time to create 1 rle with new method : 0.8889496326446533 time for calcul the mask position with numpy : 0.03741598129272461 nb_pixel_total : 40097 time to create 1 rle with old method : 0.044327497482299805 time for calcul the mask position with numpy : 0.03756904602050781 nb_pixel_total : 566 time to create 1 rle with old method : 0.0009043216705322266 time for calcul the mask position with numpy : 0.04392695426940918 nb_pixel_total : 60107 time to create 1 rle with old method : 0.0755312442779541 time for calcul the mask position with numpy : 0.037109375 nb_pixel_total : 95656 time to create 1 rle with old method : 0.10598897933959961 time for calcul the mask position with numpy : 0.035918235778808594 nb_pixel_total : 66980 time to create 1 rle with old method : 0.08145475387573242 time for calcul the mask position with numpy : 0.023967504501342773 nb_pixel_total : 1761 time to create 1 rle with old method : 0.0020275115966796875 time for calcul the mask position with numpy : 0.02542400360107422 nb_pixel_total : 3351 time to create 1 rle with old method : 0.0038738250732421875 time for calcul the mask position with numpy : 0.025183916091918945 nb_pixel_total : 19052 time to create 1 rle with old method : 0.021675586700439453 time for calcul the mask position with numpy : 0.02953028678894043 nb_pixel_total : 12493 time to create 1 rle with old method : 0.014118432998657227 time for calcul the mask position with numpy : 0.023934364318847656 nb_pixel_total : 67687 time to create 1 rle with old method : 0.07582426071166992 time for calcul the mask position with numpy : 0.023029088973999023 nb_pixel_total : 20071 time to create 1 rle with old method : 0.023835182189941406 time for calcul the mask position with numpy : 0.02733016014099121 nb_pixel_total : 153649 time to create 1 rle with new method : 0.4745769500732422 create new chi : 2.9256985187530518 time to delete rle : 0.0009205341339111328 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 8645 TO DO : save crop sub photo not yet done ! save time : 4.814286470413208 nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 10.346255540847778 time for calcul the mask position with numpy : 0.4635143280029297 nb_pixel_total : 6207894 time to create 1 rle with new method : 0.5045959949493408 time for calcul the mask position with numpy : 0.05156707763671875 nb_pixel_total : 122065 time to create 1 rle with old method : 0.19871807098388672 time for calcul the mask position with numpy : 0.051763296127319336 nb_pixel_total : 130428 time to create 1 rle with old method : 0.16716456413269043 time for calcul the mask position with numpy : 0.04085683822631836 nb_pixel_total : 10624 time to create 1 rle with old method : 0.01304173469543457 time for calcul the mask position with numpy : 0.04376506805419922 nb_pixel_total : 1499 time to create 1 rle with old method : 0.0020546913146972656 time for calcul the mask position with numpy : 0.035948991775512695 nb_pixel_total : 18670 time to create 1 rle with old method : 0.020807743072509766 time for calcul the mask position with numpy : 0.02978801727294922 nb_pixel_total : 63138 time to create 1 rle with old method : 0.07486224174499512 time for calcul the mask position with numpy : 0.02605462074279785 nb_pixel_total : 1195 time to create 1 rle with old method : 0.001436471939086914 time for calcul the mask position with numpy : 0.02389979362487793 nb_pixel_total : 17372 time to create 1 rle with old method : 0.0198361873626709 time for calcul the mask position with numpy : 0.025280237197875977 nb_pixel_total : 109852 time to create 1 rle with old method : 0.12522530555725098 time for calcul the mask position with numpy : 0.029499292373657227 nb_pixel_total : 14773 time to create 1 rle with old method : 0.016681671142578125 time for calcul the mask position with numpy : 0.0334773063659668 nb_pixel_total : 352730 time to create 1 rle with new method : 0.5914628505706787 create new chi : 2.6755576133728027 time to delete rle : 0.0011081695556640625 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 8583 TO DO : save crop sub photo not yet done ! save time : 5.219227075576782 map_output_result : {1342515556: (0.0, 'Should be the crop_list due to order', 0), 1342515358: (0.0, 'Should be the crop_list due to order', 0), 1342515354: (0.0, 'Should be the crop_list due to order', 0), 1342515351: (0.0, 'Should be the crop_list due to order', 0), 1342515347: (0.0, 'Should be the crop_list due to order', 0), 1342515343: (0.0, 'Should be the crop_list due to order', 0), 1342515340: (0.0, 'Should be the crop_list due to order', 0), 1342515147: (0.0, 'Should be the crop_list due to order', 0), 1342515144: (0.0, 'Should be the crop_list due to order', 0), 1342515140: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1342515556, 1342515358, 1342515354, 1342515351, 1342515347, 1342515343, 1342515340, 1342515147, 1342515144, 1342515140] Looping around the photos to save general results len do output : 10 /1342515556.Didn't retrieve data . /1342515358.Didn't retrieve data . /1342515354.Didn't retrieve data . /1342515351.Didn't retrieve data . /1342515347.Didn't retrieve data . /1342515343.Didn't retrieve data . /1342515340.Didn't retrieve data . /1342515147.Didn't retrieve data . /1342515144.Didn't retrieve data . /1342515140.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515556', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515358', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515354', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515351', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515347', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515343', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515147', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515144', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515140', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01316690444946289 save_final save missing photos in datou_result : time spend for datou_step_exec : 209.23061609268188 time spend to save output : 0.0156252384185791 total time spend for step 5 : 209.24624133110046 step6:crop_condition Tue Mar 11 01:01:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 155 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741651308_1837233 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.2875683307647705 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741651316_1837233 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.567908525466919 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741651322_1837233 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.569987773895264 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741651330_1837233 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9819059371948242 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741651334_1837233 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.090922832489014 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741651343_1837233 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.234975576400757 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741651349_1837233 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.6945323944091797 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1342515556, 1342515358, 1342515354, 1342515351, 1342515347, 1342515343, 1342515340, 1342515147, 1342515144, 1342515140] Looping around the photos to save general results len do output : 40 /1342738617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738618Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738619Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738622Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738829Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342738950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342739151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515556', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515358', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515354', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515351', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515347', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515343', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515147', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515144', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515140', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 130 time used for this insertion : 0.02930927276611328 save_final save missing photos in datou_result : time spend for datou_step_exec : 52.74050259590149 time spend to save output : 0.1281447410583496 total time spend for step 6 : 52.86864733695984 step7:ventilate_hashtags_in_portfolio Tue Mar 11 01:02:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 21247982 get user id for portfolio 21247982 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247982 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','Carton_gris','Teint_Dans_La_Masse','plastique','metal','flou','environnement','kraft','cartonnette','autre_refus','mal_croppe','papier')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247982 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','Carton_gris','Teint_Dans_La_Masse','plastique','metal','flou','environnement','kraft','cartonnette','autre_refus','mal_croppe','papier')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/21248250,21248251,21248252,21248253,21248254,21248255,21248256,21248257,21248258,21248259,21248260,21248261?tags=metal,Teint_Dans_La_Masse,Carton_gris,cartonnette,autre_refus,Carton_brun,environnement,flou,papier,kraft,mal_croppe,plastique Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1342515556, 1342515358, 1342515354, 1342515351, 1342515347, 1342515343, 1342515340, 1342515147, 1342515144, 1342515140] Looping around the photos to save general results len do output : 1 /21247982. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515556', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515358', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515354', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515351', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515347', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515343', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515147', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515144', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515140', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 1.7468912601470947 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.7534193992614746 time spend to save output : 1.7472074031829834 total time spend for step 7 : 4.500626802444458 step8:final Tue Mar 11 01:02:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1342515556: ('0.07563186270288548',), 1342515358: ('0.07563186270288548',), 1342515354: ('0.07563186270288548',), 1342515351: ('0.07563186270288548',), 1342515347: ('0.07563186270288548',), 1342515343: ('0.07563186270288548',), 1342515340: ('0.07563186270288548',), 1342515147: ('0.07563186270288548',), 1342515144: ('0.07563186270288548',), 1342515140: ('0.07563186270288548',)} new output for save of step final : {1342515556: ('0.07563186270288548',), 1342515358: ('0.07563186270288548',), 1342515354: ('0.07563186270288548',), 1342515351: ('0.07563186270288548',), 1342515347: ('0.07563186270288548',), 1342515343: ('0.07563186270288548',), 1342515340: ('0.07563186270288548',), 1342515147: ('0.07563186270288548',), 1342515144: ('0.07563186270288548',), 1342515140: ('0.07563186270288548',)} [1342515556, 1342515358, 1342515354, 1342515351, 1342515347, 1342515343, 1342515340, 1342515147, 1342515144, 1342515140] Looping around the photos to save general results len do output : 10 /1342515556.Didn't retrieve data . /1342515358.Didn't retrieve data . /1342515354.Didn't retrieve data . /1342515351.Didn't retrieve data . /1342515347.Didn't retrieve data . /1342515343.Didn't retrieve data . /1342515340.Didn't retrieve data . /1342515147.Didn't retrieve data . /1342515144.Didn't retrieve data . /1342515140.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515556', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515358', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515354', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515351', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515347', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515343', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515147', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515144', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515140', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.029107093811035156 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.13444948196411133 time spend to save output : 0.029723405838012695 total time spend for step 8 : 0.16417288780212402 step9:velours_tree Tue Mar 11 01:02:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.2963745594024658 time spend to save output : 4.8160552978515625e-05 total time spend for step 9 : 0.29642271995544434 step10:send_mail_cod Tue Mar 11 01:02:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P21247982_11-03-2025_01_02_41.pdf 21248250 imagette212482501741651361 21248253 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482531741651361 21248254 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482541741651361 21248257 imagette212482571741651363 21248259 change filename to text .change filename to text .imagette212482591741651363 21248260 imagette212482601741651363 21248261 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482611741651363 21248251 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482511741651364 21248252 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482521741651366 21248255 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482551741651367 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21247982 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21248250,21248251,21248252,21248253,21248254,21248255,21248256,21248257,21248258,21248259,21248260,21248261?tags=metal,Teint_Dans_La_Masse,Carton_gris,cartonnette,autre_refus,Carton_brun,environnement,flou,papier,kraft,mal_croppe,plastique your option no_mail is active, we will not send the real mail to your client args[1342515556] : ('0.07563186270288548',) no score found for photo 1342515556 We are sending mail with results at report@fotonower.com args[1342515358] : ('0.07563186270288548',) no score found for photo 1342515358 We are sending mail with results at report@fotonower.com args[1342515354] : ('0.07563186270288548',) no score found for photo 1342515354 We are sending mail with results at report@fotonower.com args[1342515351] : ('0.07563186270288548',) no score found for photo 1342515351 We are sending mail with results at report@fotonower.com args[1342515347] : ('0.07563186270288548',) no score found for photo 1342515347 We are sending mail with results at report@fotonower.com args[1342515343] : ('0.07563186270288548',) no score found for photo 1342515343 We are sending mail with results at report@fotonower.com args[1342515340] : ('0.07563186270288548',) no score found for photo 1342515340 We are sending mail with results at report@fotonower.com args[1342515147] : ('0.07563186270288548',) no score found for photo 1342515147 We are sending mail with results at report@fotonower.com args[1342515144] : ('0.07563186270288548',) no score found for photo 1342515144 We are sending mail with results at report@fotonower.com args[1342515140] : ('0.07563186270288548',) no score found for photo 1342515140 We are sending mail with results at report@fotonower.com refus_total : 0.07563186270288548 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21247982 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1342515634,1342684754,1342684711,1342684707,1342684703,1342684467,1342684427,1342684340,1342684297,1342515140,1342515556,1342515351,1342515347,1342515343,1342515147,1342515144,1342515340,1342515358,1342515354,1342684741) Found this number of photos: 20 begin to download photo : 1342515634 begin to download photo : 1342684467 begin to download photo : 1342515556 begin to download photo : 1342515144 download finish for photo 1342684467 begin to download photo : 1342684427 download finish for photo 1342515144 begin to download photo : 1342515340 download finish for photo 1342515634 begin to download photo : 1342684754 download finish for photo 1342515556 begin to download photo : 1342515351 download finish for photo 1342515340 begin to download photo : 1342515358 download finish for photo 1342684754 begin to download photo : 1342684711 download finish for photo 1342684427 begin to download photo : 1342684340 download finish for photo 1342515358 begin to download photo : 1342515354 download finish for photo 1342515351 begin to download photo : 1342515347 download finish for photo 1342684711 begin to download photo : 1342684707 download finish for photo 1342515354 begin to download photo : 1342684741 download finish for photo 1342684340 begin to download photo : 1342684297 download finish for photo 1342515347 begin to download photo : 1342515343 download finish for photo 1342684741 download finish for photo 1342684297 begin to download photo : 1342515140 download finish for photo 1342684707 begin to download photo : 1342684703 download finish for photo 1342515343 begin to download photo : 1342515147 download finish for photo 1342684703 download finish for photo 1342515140 download finish for photo 1342515147 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247982_11-03-2025_01_02_41.pdf results_Auto_P21247982_11-03-2025_01_02_41.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247982_11-03-2025_01_02_41.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','21247982','results_Auto_P21247982_11-03-2025_01_02_41.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247982_11-03-2025_01_02_41.pdf','pdf','','1.22','0.07563186270288548') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1342515556, 1342515358, 1342515354, 1342515351, 1342515347, 1342515343, 1342515340, 1342515147, 1342515144, 1342515140] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515556', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515358', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515354', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515351', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515347', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515343', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515147', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515144', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515140', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.019336223602294922 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.570375204086304 time spend to save output : 0.019614696502685547 total time spend for step 10 : 12.58998990058899 step11:split_time_score Tue Mar 11 01:02:53 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 20),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10032025 21247982 Nombre de photos uploadées : 20 / 23040 (0%) 10032025 21247982 Nombre de photos taguées (types de déchets): 0 / 20 (0%) 10032025 21247982 Nombre de photos taguées (volume) : 0 / 20 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 8.106231689453125e-06 ???????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0009865760803222656 elapsed_time : insert_dashboard_record_day_entry 0.03156232833862305 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244281 order by id desc limit 1 Qualite : 0.20374319704010083 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244282_10-03-2025_23_03_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244282 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244282 AND mptpi.`type`=3594 To do Qualite : 0.18168180090323166 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244283_10-03-2025_22_41_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244283 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244283 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244308_11-03-2025_00_50_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244308 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244309_11-03-2025_00_54_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244309 order by id desc limit 1 Qualite : 0.34839745750897183 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244310_10-03-2025_23_16_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244310 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244310 AND mptpi.`type`=4200 To do Qualite : 0.2737082523489908 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244311_10-03-2025_22_37_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244311 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244311 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247979 order by id desc limit 1 Qualite : 0.06524887220116574 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245755_10-03-2025_23_29_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245755 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245755 AND mptpi.`type`=4207 To do Qualite : 0.072106598848494 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245772_10-03-2025_23_26_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245772 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245772 AND mptpi.`type`=4207 To do Qualite : 0.04447951517284577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245773_10-03-2025_23_08_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245773 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245773 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247980 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247981 order by id desc limit 1 Qualite : 0.07617732921344036 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247982_11-03-2025_01_02_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247982 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247982 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247983 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247984 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247985 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247986 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247987 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247988 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247989 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247990 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247991 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247992 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247993 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222608 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222609 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222611 order by id desc limit 1 Qualite : 0.04897866899276895 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21222613_10-03-2025_17_51_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222613 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222613 AND mptpi.`type`=4209 To do Qualite : 0.05516057268644328 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21222058_10-03-2025_14_57_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222058 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222058 AND mptpi.`type`=3726 To do Qualite : 0.2309571105130506 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21222084_10-03-2025_15_14_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222084 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222084 AND mptpi.`type`=3594 To do Qualite : 0.07631761142738193 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247994_11-03-2025_01_02_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247994 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247994 AND mptpi.`type`=4211 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247995 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247996 order by id desc limit 1 Qualite : 0.09073664900560331 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247997_11-03-2025_00_59_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247997 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247997 AND mptpi.`type`=4207 To do Qualite : 0.04018401839319672 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247998_11-03-2025_01_02_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247998 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247998 AND mptpi.`type`=4207 To do Qualite : 0.06957109304578478 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247999_11-03-2025_00_53_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247999 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247999 AND mptpi.`type`=4207 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21248000_11-03-2025_00_59_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248000 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21248001_11-03-2025_00_56_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248001 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248014 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21248019_11-03-2025_00_52_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248019 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248020 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248021 order by id desc limit 1 Qualite : 0.08314390126051041 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245775_10-03-2025_23_04_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245775 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245775 AND mptpi.`type`=3726 To do Qualite : 0.07125952811625422 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245776_10-03-2025_23_20_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245776 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245776 AND mptpi.`type`=4209 To do Qualite : 0.08640782218121312 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245777_10-03-2025_23_45_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245777 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245777 AND mptpi.`type`=4207 To do Qualite : 0.05258407715521776 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245779_10-03-2025_23_16_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245779 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245779 AND mptpi.`type`=4207 To do Qualite : 0.06963896934644463 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245780_10-03-2025_22_56_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245780 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245780 AND mptpi.`type`=4207 To do Qualite : 0.06276088819330108 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245781_10-03-2025_23_01_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245781 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245781 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21236129 order by id desc limit 1 Qualite : 0.2393178491409189 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21236155_10-03-2025_21_12_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21236155 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21236155 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21236158_10-03-2025_19_45_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21236158 order by id desc limit 1 Qualite : 0.2502752435850885 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21236177_10-03-2025_19_58_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21236177 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21236177 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248022 order by id desc limit 1 Qualite : 0.22092496141975312 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244378_10-03-2025_23_40_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244378 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244378 AND mptpi.`type`=3594 To do Qualite : 0.0657541186557689 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244384_11-03-2025_00_34_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244384 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244384 AND mptpi.`type`=4207 To do Qualite : 0.05471434328258402 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244385_10-03-2025_22_31_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244385 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244385 AND mptpi.`type`=4207 To do Qualite : 0.05735829306752358 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244386_10-03-2025_22_27_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244386 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244386 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10032025': {'nb_upload': 20, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1342515556, 1342515358, 1342515354, 1342515351, 1342515347, 1342515343, 1342515340, 1342515147, 1342515144, 1342515140] Looping around the photos to save general results len do output : 1 /21247982Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515556', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515358', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515354', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515351', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515347', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515343', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515147', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515144', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', None, '1342515140', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.21367597579956055 save_final save missing photos in datou_result : time spend for datou_step_exec : 25.024733781814575 time spend to save output : 0.21403002738952637 total time spend for step 11 : 25.2387638092041 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 153.62user 180.28system 8:57.07elapsed 62%CPU (0avgtext+0avgdata 4075532maxresident)k 3502328inputs+149896outputs (43972major+12215327minor)pagefaults 0swaps