python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1660800 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 11>10 please execute split_portfolio.py -i 21454786 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2671130'] with mtr_portfolio_ids : ['21454786'] and first list_photo_ids : [] new path : /proc/1660800/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.4727070331573486 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Fri Mar 14 22:26:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 1792 wait 20 seconds l 3637 free memory gpu now : 1792 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-14 22:26:52.097902: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-14 22:26:52.123212: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-14 22:26:52.126130: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f3fb4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-14 22:26:52.126187: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-14 22:26:52.130051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-14 22:26:52.369549: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3dc899a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-14 22:26:52.369592: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-14 22:26:52.370659: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-14 22:26:52.371038: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-14 22:26:52.373816: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-14 22:26:52.376465: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-14 22:26:52.376931: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-14 22:26:52.380175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-14 22:26:52.381460: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-14 22:26:52.386439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-14 22:26:52.387682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-14 22:26:52.387750: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-14 22:26:52.388325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-14 22:26:52.388339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-14 22:26:52.388348: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-14 22:26:52.389304: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3438 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-14 22:26:52.653333: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-14 22:26:52.653475: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-14 22:26:52.653561: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-14 22:26:52.653599: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-14 22:26:52.653628: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-14 22:26:52.653656: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-14 22:26:52.653683: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-14 22:26:52.653734: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-14 22:26:52.655002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-14 22:26:52.656058: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-14 22:26:52.656095: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-14 22:26:52.656116: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-14 22:26:52.656137: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-14 22:26:52.656156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-14 22:26:52.656176: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-14 22:26:52.656195: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-14 22:26:52.656215: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-14 22:26:52.657110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-14 22:26:52.657136: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-14 22:26:52.657145: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-14 22:26:52.657153: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-14 22:26:52.658067: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3438 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-14 22:27:04.651687: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-14 22:27:04.869450: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-14 22:27:06.503291: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.242489: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-03-14 22:27:07.322458: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.322534: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.323709: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.323742: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.67GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.330763: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.330792: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.29GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.337033: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.337052: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.337626: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.337641: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.78GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.384926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.384985: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.385572: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.385587: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 63.85MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.386177: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.386192: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.386774: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.386788: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.26GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-14 22:27:07.394600: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.395198: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.411827: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.412656: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.413527: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.414389: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.418452: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.419052: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.419682: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.420312: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.421302: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.421317: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-03-14 22:27:07.431497: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.432080: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.440931: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.441511: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.442095: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.442670: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.443272: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-14 22:27:07.443850: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.86G (3070099456 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 31 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 10 Detection mask done ! Trying to reset tf kernel 1662026 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2697 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3890 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.005667448043823242 nb_pixel_total : 177379 time to create 1 rle with new method : 0.013088464736938477 length of segment : 447 time for calcul the mask position with numpy : 0.00012230873107910156 nb_pixel_total : 1777 time to create 1 rle with old method : 0.0027370452880859375 length of segment : 58 time for calcul the mask position with numpy : 0.0005319118499755859 nb_pixel_total : 22163 time to create 1 rle with old method : 0.03339791297912598 length of segment : 182 time for calcul the mask position with numpy : 0.0014023780822753906 nb_pixel_total : 45253 time to create 1 rle with old method : 0.062185049057006836 length of segment : 294 time for calcul the mask position with numpy : 0.0013933181762695312 nb_pixel_total : 55016 time to create 1 rle with old method : 0.07141947746276855 length of segment : 315 time for calcul the mask position with numpy : 0.0006346702575683594 nb_pixel_total : 22974 time to create 1 rle with old method : 0.030209064483642578 length of segment : 297 time for calcul the mask position with numpy : 8.654594421386719e-05 nb_pixel_total : 2226 time to create 1 rle with old method : 0.0029146671295166016 length of segment : 64 time for calcul the mask position with numpy : 0.0009434223175048828 nb_pixel_total : 56958 time to create 1 rle with old method : 0.07078146934509277 length of segment : 283 time for calcul the mask position with numpy : 0.00018310546875 nb_pixel_total : 3583 time to create 1 rle with old method : 0.006387948989868164 length of segment : 119 time for calcul the mask position with numpy : 0.0006296634674072266 nb_pixel_total : 23115 time to create 1 rle with old method : 0.03720998764038086 length of segment : 204 time for calcul the mask position with numpy : 0.002377748489379883 nb_pixel_total : 51623 time to create 1 rle with old method : 0.0696713924407959 length of segment : 441 time for calcul the mask position with numpy : 0.0010645389556884766 nb_pixel_total : 46834 time to create 1 rle with old method : 0.05500292778015137 length of segment : 420 time for calcul the mask position with numpy : 0.00010776519775390625 nb_pixel_total : 1039 time to create 1 rle with old method : 0.0018279552459716797 length of segment : 57 time for calcul the mask position with numpy : 0.00014519691467285156 nb_pixel_total : 4112 time to create 1 rle with old method : 0.005182027816772461 length of segment : 94 time for calcul the mask position with numpy : 0.001689910888671875 nb_pixel_total : 56042 time to create 1 rle with old method : 0.07544684410095215 length of segment : 307 time for calcul the mask position with numpy : 0.00015401840209960938 nb_pixel_total : 2447 time to create 1 rle with old method : 0.004047393798828125 length of segment : 40 time for calcul the mask position with numpy : 0.00031375885009765625 nb_pixel_total : 11175 time to create 1 rle with old method : 0.015824317932128906 length of segment : 125 time for calcul the mask position with numpy : 0.1169281005859375 nb_pixel_total : 357892 time to create 1 rle with new method : 0.03233075141906738 length of segment : 962 time for calcul the mask position with numpy : 0.000385284423828125 nb_pixel_total : 7994 time to create 1 rle with old method : 0.017015457153320312 length of segment : 188 time for calcul the mask position with numpy : 0.0014333724975585938 nb_pixel_total : 87204 time to create 1 rle with old method : 0.1186227798461914 length of segment : 382 time for calcul the mask position with numpy : 0.0003216266632080078 nb_pixel_total : 14899 time to create 1 rle with old method : 0.018168926239013672 length of segment : 289 time for calcul the mask position with numpy : 0.000194549560546875 nb_pixel_total : 8690 time to create 1 rle with old method : 0.010387182235717773 length of segment : 82 time for calcul the mask position with numpy : 8.821487426757812e-05 nb_pixel_total : 908 time to create 1 rle with old method : 0.0016222000122070312 length of segment : 30 time for calcul the mask position with numpy : 0.0001285076141357422 nb_pixel_total : 5540 time to create 1 rle with old method : 0.0074045658111572266 length of segment : 90 time for calcul the mask position with numpy : 0.003747224807739258 nb_pixel_total : 275392 time to create 1 rle with new method : 0.01418614387512207 length of segment : 723 time for calcul the mask position with numpy : 0.00023221969604492188 nb_pixel_total : 10687 time to create 1 rle with old method : 0.013228893280029297 length of segment : 152 time for calcul the mask position with numpy : 0.0008349418640136719 nb_pixel_total : 32464 time to create 1 rle with old method : 0.04783987998962402 length of segment : 390 time for calcul the mask position with numpy : 0.0005846023559570312 nb_pixel_total : 35346 time to create 1 rle with old method : 0.045366525650024414 length of segment : 208 time for calcul the mask position with numpy : 0.0008065700531005859 nb_pixel_total : 43576 time to create 1 rle with old method : 0.05301690101623535 length of segment : 361 time for calcul the mask position with numpy : 0.0002999305725097656 nb_pixel_total : 15858 time to create 1 rle with old method : 0.018904685974121094 length of segment : 115 time for calcul the mask position with numpy : 0.0002930164337158203 nb_pixel_total : 12022 time to create 1 rle with old method : 0.014673471450805664 length of segment : 155 time for calcul the mask position with numpy : 0.0002818107604980469 nb_pixel_total : 2710 time to create 1 rle with old method : 0.0033864974975585938 length of segment : 113 time for calcul the mask position with numpy : 0.0003323554992675781 nb_pixel_total : 9066 time to create 1 rle with old method : 0.01150369644165039 length of segment : 121 time for calcul the mask position with numpy : 0.0013272762298583984 nb_pixel_total : 28604 time to create 1 rle with old method : 0.04560112953186035 length of segment : 210 time for calcul the mask position with numpy : 0.0007503032684326172 nb_pixel_total : 12597 time to create 1 rle with old method : 0.015198469161987305 length of segment : 161 time for calcul the mask position with numpy : 0.008322954177856445 nb_pixel_total : 300177 time to create 1 rle with new method : 0.01696944236755371 length of segment : 717 time for calcul the mask position with numpy : 0.0005273818969726562 nb_pixel_total : 8610 time to create 1 rle with old method : 0.009952068328857422 length of segment : 145 time for calcul the mask position with numpy : 0.003057241439819336 nb_pixel_total : 95072 time to create 1 rle with old method : 0.10961461067199707 length of segment : 413 time for calcul the mask position with numpy : 0.0014352798461914062 nb_pixel_total : 24776 time to create 1 rle with old method : 0.02801990509033203 length of segment : 247 time for calcul the mask position with numpy : 0.0058515071868896484 nb_pixel_total : 94829 time to create 1 rle with old method : 0.10776233673095703 length of segment : 579 time for calcul the mask position with numpy : 0.00019741058349609375 nb_pixel_total : 2089 time to create 1 rle with old method : 0.002599954605102539 length of segment : 59 time for calcul the mask position with numpy : 0.002738475799560547 nb_pixel_total : 95891 time to create 1 rle with old method : 0.11321616172790527 length of segment : 472 time for calcul the mask position with numpy : 0.0002155303955078125 nb_pixel_total : 2224 time to create 1 rle with old method : 0.0027136802673339844 length of segment : 63 time for calcul the mask position with numpy : 0.001546621322631836 nb_pixel_total : 98397 time to create 1 rle with old method : 0.1162865161895752 length of segment : 504 time for calcul the mask position with numpy : 0.004861593246459961 nb_pixel_total : 149552 time to create 1 rle with old method : 0.17734336853027344 length of segment : 584 time for calcul the mask position with numpy : 0.0003514289855957031 nb_pixel_total : 10008 time to create 1 rle with old method : 0.011476993560791016 length of segment : 159 time for calcul the mask position with numpy : 0.0030426979064941406 nb_pixel_total : 82545 time to create 1 rle with old method : 0.09762287139892578 length of segment : 460 time for calcul the mask position with numpy : 0.0026814937591552734 nb_pixel_total : 149296 time to create 1 rle with old method : 0.18249154090881348 length of segment : 575 time for calcul the mask position with numpy : 0.00040268898010253906 nb_pixel_total : 13588 time to create 1 rle with old method : 0.016292095184326172 length of segment : 119 time for calcul the mask position with numpy : 0.00023055076599121094 nb_pixel_total : 2923 time to create 1 rle with old method : 0.0036270618438720703 length of segment : 54 time for calcul the mask position with numpy : 0.00043201446533203125 nb_pixel_total : 16696 time to create 1 rle with old method : 0.020426034927368164 length of segment : 156 time for calcul the mask position with numpy : 0.0005605220794677734 nb_pixel_total : 16988 time to create 1 rle with old method : 0.022460222244262695 length of segment : 186 time for calcul the mask position with numpy : 0.0003829002380371094 nb_pixel_total : 6617 time to create 1 rle with old method : 0.011297941207885742 length of segment : 92 time for calcul the mask position with numpy : 0.0003781318664550781 nb_pixel_total : 17490 time to create 1 rle with old method : 0.021181583404541016 length of segment : 272 time for calcul the mask position with numpy : 0.0005435943603515625 nb_pixel_total : 7931 time to create 1 rle with old method : 0.009180307388305664 length of segment : 252 time for calcul the mask position with numpy : 0.001435995101928711 nb_pixel_total : 31933 time to create 1 rle with old method : 0.03913140296936035 length of segment : 318 time for calcul the mask position with numpy : 0.0036802291870117188 nb_pixel_total : 152249 time to create 1 rle with new method : 0.013129234313964844 length of segment : 595 time for calcul the mask position with numpy : 0.0002834796905517578 nb_pixel_total : 1662 time to create 1 rle with old method : 0.0021886825561523438 length of segment : 37 time for calcul the mask position with numpy : 0.0013697147369384766 nb_pixel_total : 79991 time to create 1 rle with old method : 0.11470961570739746 length of segment : 343 time for calcul the mask position with numpy : 0.002864360809326172 nb_pixel_total : 97198 time to create 1 rle with old method : 0.12630414962768555 length of segment : 592 time for calcul the mask position with numpy : 0.0004017353057861328 nb_pixel_total : 11564 time to create 1 rle with old method : 0.014722347259521484 length of segment : 97 time for calcul the mask position with numpy : 0.00041747093200683594 nb_pixel_total : 14887 time to create 1 rle with old method : 0.01852583885192871 length of segment : 141 time for calcul the mask position with numpy : 0.004212856292724609 nb_pixel_total : 182656 time to create 1 rle with new method : 0.014099597930908203 length of segment : 768 time for calcul the mask position with numpy : 0.003234386444091797 nb_pixel_total : 156213 time to create 1 rle with new method : 0.008615970611572266 length of segment : 573 time for calcul the mask position with numpy : 0.00019788742065429688 nb_pixel_total : 1902 time to create 1 rle with old method : 0.0028841495513916016 length of segment : 44 time for calcul the mask position with numpy : 0.0006120204925537109 nb_pixel_total : 27399 time to create 1 rle with old method : 0.0338129997253418 length of segment : 206 time for calcul the mask position with numpy : 0.0007131099700927734 nb_pixel_total : 8114 time to create 1 rle with old method : 0.010453939437866211 length of segment : 282 time for calcul the mask position with numpy : 0.0003063678741455078 nb_pixel_total : 8194 time to create 1 rle with old method : 0.01016092300415039 length of segment : 62 time for calcul the mask position with numpy : 0.002178668975830078 nb_pixel_total : 56557 time to create 1 rle with old method : 0.0682840347290039 length of segment : 467 time for calcul the mask position with numpy : 0.0005700588226318359 nb_pixel_total : 7412 time to create 1 rle with old method : 0.009236812591552734 length of segment : 303 time for calcul the mask position with numpy : 0.0006611347198486328 nb_pixel_total : 38638 time to create 1 rle with old method : 0.045722246170043945 length of segment : 236 time for calcul the mask position with numpy : 0.0035736560821533203 nb_pixel_total : 106236 time to create 1 rle with old method : 0.14284849166870117 length of segment : 557 time for calcul the mask position with numpy : 0.00023245811462402344 nb_pixel_total : 2745 time to create 1 rle with old method : 0.0035119056701660156 length of segment : 54 time for calcul the mask position with numpy : 0.0017642974853515625 nb_pixel_total : 36540 time to create 1 rle with old method : 0.0432887077331543 length of segment : 282 time for calcul the mask position with numpy : 0.0007190704345703125 nb_pixel_total : 16485 time to create 1 rle with old method : 0.01920938491821289 length of segment : 131 time for calcul the mask position with numpy : 0.0023009777069091797 nb_pixel_total : 58701 time to create 1 rle with old method : 0.06694698333740234 length of segment : 432 time for calcul the mask position with numpy : 0.0022809505462646484 nb_pixel_total : 70976 time to create 1 rle with old method : 0.08693671226501465 length of segment : 367 time for calcul the mask position with numpy : 0.0008141994476318359 nb_pixel_total : 17781 time to create 1 rle with old method : 0.021489858627319336 length of segment : 238 time for calcul the mask position with numpy : 0.0012483596801757812 nb_pixel_total : 70536 time to create 1 rle with old method : 0.08147192001342773 length of segment : 330 time for calcul the mask position with numpy : 0.002054929733276367 nb_pixel_total : 82819 time to create 1 rle with old method : 0.09254860877990723 length of segment : 467 time for calcul the mask position with numpy : 0.0026006698608398438 nb_pixel_total : 80429 time to create 1 rle with old method : 0.10015439987182617 length of segment : 235 time for calcul the mask position with numpy : 0.0006153583526611328 nb_pixel_total : 15753 time to create 1 rle with old method : 0.018548965454101562 length of segment : 152 time for calcul the mask position with numpy : 0.002383708953857422 nb_pixel_total : 88631 time to create 1 rle with old method : 0.11244368553161621 length of segment : 379 time for calcul the mask position with numpy : 0.0026886463165283203 nb_pixel_total : 89201 time to create 1 rle with old method : 0.10513424873352051 length of segment : 502 time for calcul the mask position with numpy : 0.0009057521820068359 nb_pixel_total : 22292 time to create 1 rle with old method : 0.026342153549194336 length of segment : 232 time for calcul the mask position with numpy : 0.0002415180206298828 nb_pixel_total : 3833 time to create 1 rle with old method : 0.004812479019165039 length of segment : 66 time for calcul the mask position with numpy : 0.006843090057373047 nb_pixel_total : 248140 time to create 1 rle with new method : 0.013479471206665039 length of segment : 549 time for calcul the mask position with numpy : 0.0019469261169433594 nb_pixel_total : 32181 time to create 1 rle with old method : 0.0377960205078125 length of segment : 502 time for calcul the mask position with numpy : 0.0029616355895996094 nb_pixel_total : 122032 time to create 1 rle with old method : 0.1519763469696045 length of segment : 403 time for calcul the mask position with numpy : 0.06529498100280762 nb_pixel_total : 1151257 time to create 1 rle with new method : 0.29711461067199707 length of segment : 2515 time for calcul the mask position with numpy : 0.01847982406616211 nb_pixel_total : 893728 time to create 1 rle with new method : 0.06602811813354492 length of segment : 1842 time for calcul the mask position with numpy : 0.0027768611907958984 nb_pixel_total : 185507 time to create 1 rle with new method : 0.00784444808959961 length of segment : 690 time for calcul the mask position with numpy : 0.0038366317749023438 nb_pixel_total : 188051 time to create 1 rle with new method : 0.011201143264770508 length of segment : 1011 time for calcul the mask position with numpy : 0.0029587745666503906 nb_pixel_total : 100766 time to create 1 rle with old method : 0.12057042121887207 length of segment : 436 time for calcul the mask position with numpy : 0.003375530242919922 nb_pixel_total : 133468 time to create 1 rle with old method : 0.1612091064453125 length of segment : 462 time for calcul the mask position with numpy : 0.017978906631469727 nb_pixel_total : 461388 time to create 1 rle with new method : 0.04542660713195801 length of segment : 832 time for calcul the mask position with numpy : 0.0030717849731445312 nb_pixel_total : 150912 time to create 1 rle with new method : 0.00759124755859375 length of segment : 577 time for calcul the mask position with numpy : 0.0026412010192871094 nb_pixel_total : 72991 time to create 1 rle with old method : 0.09073567390441895 length of segment : 471 time for calcul the mask position with numpy : 0.0068569183349609375 nb_pixel_total : 286635 time to create 1 rle with new method : 0.010896444320678711 length of segment : 759 time for calcul the mask position with numpy : 0.0024194717407226562 nb_pixel_total : 75173 time to create 1 rle with old method : 0.09133410453796387 length of segment : 455 time for calcul the mask position with numpy : 0.0023195743560791016 nb_pixel_total : 116154 time to create 1 rle with old method : 0.15114998817443848 length of segment : 471 time for calcul the mask position with numpy : 0.003172159194946289 nb_pixel_total : 133211 time to create 1 rle with old method : 0.17046403884887695 length of segment : 462 time for calcul the mask position with numpy : 0.00180816650390625 nb_pixel_total : 47455 time to create 1 rle with old method : 0.057216644287109375 length of segment : 412 time for calcul the mask position with numpy : 0.0040340423583984375 nb_pixel_total : 272459 time to create 1 rle with new method : 0.010165929794311523 length of segment : 688 time for calcul the mask position with numpy : 0.0016596317291259766 nb_pixel_total : 72491 time to create 1 rle with old method : 0.0858001708984375 length of segment : 425 time for calcul the mask position with numpy : 0.004062652587890625 nb_pixel_total : 217809 time to create 1 rle with new method : 0.006079673767089844 length of segment : 464 time for calcul the mask position with numpy : 0.03690624237060547 nb_pixel_total : 1320584 time to create 1 rle with new method : 0.35101962089538574 length of segment : 1646 time for calcul the mask position with numpy : 0.0004458427429199219 nb_pixel_total : 19653 time to create 1 rle with old method : 0.02280116081237793 length of segment : 223 time for calcul the mask position with numpy : 0.005799055099487305 nb_pixel_total : 360685 time to create 1 rle with new method : 0.011194944381713867 length of segment : 1182 time for calcul the mask position with numpy : 0.00013589859008789062 nb_pixel_total : 2792 time to create 1 rle with old method : 0.0036585330963134766 length of segment : 61 time for calcul the mask position with numpy : 0.006348609924316406 nb_pixel_total : 261110 time to create 1 rle with new method : 0.016019344329833984 length of segment : 686 time for calcul the mask position with numpy : 0.000423431396484375 nb_pixel_total : 24876 time to create 1 rle with old method : 0.028658628463745117 length of segment : 173 time for calcul the mask position with numpy : 0.0011546611785888672 nb_pixel_total : 88476 time to create 1 rle with old method : 0.10253620147705078 length of segment : 246 time for calcul the mask position with numpy : 0.00790715217590332 nb_pixel_total : 498595 time to create 1 rle with new method : 0.020900249481201172 length of segment : 887 time for calcul the mask position with numpy : 0.008914709091186523 nb_pixel_total : 561052 time to create 1 rle with new method : 0.0184938907623291 length of segment : 1123 time for calcul the mask position with numpy : 0.00013136863708496094 nb_pixel_total : 3381 time to create 1 rle with old method : 0.003950357437133789 length of segment : 59 time for calcul the mask position with numpy : 0.00629878044128418 nb_pixel_total : 206434 time to create 1 rle with new method : 0.010331153869628906 length of segment : 703 time for calcul the mask position with numpy : 0.0006928443908691406 nb_pixel_total : 11090 time to create 1 rle with old method : 0.014242887496948242 length of segment : 226 time for calcul the mask position with numpy : 0.0017819404602050781 nb_pixel_total : 34979 time to create 1 rle with old method : 0.04713177680969238 length of segment : 388 time for calcul the mask position with numpy : 0.002765655517578125 nb_pixel_total : 90016 time to create 1 rle with old method : 0.11164665222167969 length of segment : 398 time for calcul the mask position with numpy : 0.0015056133270263672 nb_pixel_total : 64852 time to create 1 rle with old method : 0.08228325843811035 length of segment : 292 time for calcul the mask position with numpy : 0.02325892448425293 nb_pixel_total : 823135 time to create 1 rle with new method : 0.09089469909667969 length of segment : 1416 time for calcul the mask position with numpy : 0.0017528533935546875 nb_pixel_total : 107741 time to create 1 rle with old method : 0.1194765567779541 length of segment : 332 time for calcul the mask position with numpy : 0.001961946487426758 nb_pixel_total : 53661 time to create 1 rle with old method : 0.057805776596069336 length of segment : 255 time for calcul the mask position with numpy : 0.0005328655242919922 nb_pixel_total : 7382 time to create 1 rle with old method : 0.008671998977661133 length of segment : 146 time for calcul the mask position with numpy : 0.004149675369262695 nb_pixel_total : 176945 time to create 1 rle with new method : 0.006471157073974609 length of segment : 597 time for calcul the mask position with numpy : 0.0003001689910888672 nb_pixel_total : 7797 time to create 1 rle with old method : 0.00942683219909668 length of segment : 57 time for calcul the mask position with numpy : 0.00042247772216796875 nb_pixel_total : 10595 time to create 1 rle with old method : 0.012339353561401367 length of segment : 155 time for calcul the mask position with numpy : 0.0005860328674316406 nb_pixel_total : 21603 time to create 1 rle with old method : 0.023888349533081055 length of segment : 195 time for calcul the mask position with numpy : 0.003820657730102539 nb_pixel_total : 145693 time to create 1 rle with old method : 0.18204426765441895 length of segment : 441 time for calcul the mask position with numpy : 0.00012826919555664062 nb_pixel_total : 2388 time to create 1 rle with old method : 0.003060579299926758 length of segment : 56 time for calcul the mask position with numpy : 0.00024080276489257812 nb_pixel_total : 8448 time to create 1 rle with old method : 0.01070547103881836 length of segment : 127 time for calcul the mask position with numpy : 0.0035135746002197266 nb_pixel_total : 81827 time to create 1 rle with old method : 0.09819293022155762 length of segment : 446 time for calcul the mask position with numpy : 0.0020139217376708984 nb_pixel_total : 116681 time to create 1 rle with old method : 0.1340956687927246 length of segment : 269 time for calcul the mask position with numpy : 0.0014019012451171875 nb_pixel_total : 44153 time to create 1 rle with old method : 0.050875186920166016 length of segment : 303 time for calcul the mask position with numpy : 0.003211498260498047 nb_pixel_total : 113512 time to create 1 rle with old method : 0.12733960151672363 length of segment : 369 time for calcul the mask position with numpy : 0.002810239791870117 nb_pixel_total : 65881 time to create 1 rle with old method : 0.07788610458374023 length of segment : 389 time for calcul the mask position with numpy : 0.00039315223693847656 nb_pixel_total : 14920 time to create 1 rle with old method : 0.018479347229003906 length of segment : 88 time for calcul the mask position with numpy : 0.0037474632263183594 nb_pixel_total : 105499 time to create 1 rle with old method : 0.1221466064453125 length of segment : 417 time for calcul the mask position with numpy : 0.0022144317626953125 nb_pixel_total : 55434 time to create 1 rle with old method : 0.06410813331604004 length of segment : 270 time for calcul the mask position with numpy : 0.0006833076477050781 nb_pixel_total : 46296 time to create 1 rle with old method : 0.0552980899810791 length of segment : 239 time for calcul the mask position with numpy : 0.0006558895111083984 nb_pixel_total : 14381 time to create 1 rle with old method : 0.016556739807128906 length of segment : 156 time for calcul the mask position with numpy : 0.002058744430541992 nb_pixel_total : 54956 time to create 1 rle with old method : 0.06780076026916504 length of segment : 570 time for calcul the mask position with numpy : 0.0014002323150634766 nb_pixel_total : 38892 time to create 1 rle with old method : 0.0471796989440918 length of segment : 382 time for calcul the mask position with numpy : 0.0005099773406982422 nb_pixel_total : 6015 time to create 1 rle with old method : 0.0076601505279541016 length of segment : 127 time for calcul the mask position with numpy : 0.008561134338378906 nb_pixel_total : 267506 time to create 1 rle with new method : 0.013027429580688477 length of segment : 758 time for calcul the mask position with numpy : 0.003034353256225586 nb_pixel_total : 126374 time to create 1 rle with old method : 0.13783502578735352 length of segment : 514 time for calcul the mask position with numpy : 0.0010082721710205078 nb_pixel_total : 26603 time to create 1 rle with old method : 0.031197071075439453 length of segment : 239 time for calcul the mask position with numpy : 0.004973173141479492 nb_pixel_total : 176676 time to create 1 rle with new method : 0.008545875549316406 length of segment : 500 time for calcul the mask position with numpy : 0.00036525726318359375 nb_pixel_total : 4671 time to create 1 rle with old method : 0.005743741989135742 length of segment : 89 time for calcul the mask position with numpy : 0.0004909038543701172 nb_pixel_total : 10150 time to create 1 rle with old method : 0.012274980545043945 length of segment : 116 time for calcul the mask position with numpy : 0.0006046295166015625 nb_pixel_total : 16904 time to create 1 rle with old method : 0.021190404891967773 length of segment : 57 time for calcul the mask position with numpy : 0.0003139972686767578 nb_pixel_total : 7045 time to create 1 rle with old method : 0.00879216194152832 length of segment : 117 time for calcul the mask position with numpy : 0.0001289844512939453 nb_pixel_total : 1855 time to create 1 rle with old method : 0.0024607181549072266 length of segment : 58 time for calcul the mask position with numpy : 0.0017695426940917969 nb_pixel_total : 68865 time to create 1 rle with old method : 0.07942318916320801 length of segment : 334 time for calcul the mask position with numpy : 0.0002865791320800781 nb_pixel_total : 8264 time to create 1 rle with old method : 0.010299205780029297 length of segment : 92 time for calcul the mask position with numpy : 0.00019812583923339844 nb_pixel_total : 3603 time to create 1 rle with old method : 0.004345893859863281 length of segment : 71 time for calcul the mask position with numpy : 0.0016162395477294922 nb_pixel_total : 34451 time to create 1 rle with old method : 0.03889870643615723 length of segment : 267 time for calcul the mask position with numpy : 0.0005254745483398438 nb_pixel_total : 16722 time to create 1 rle with old method : 0.02208876609802246 length of segment : 199 time for calcul the mask position with numpy : 0.0012631416320800781 nb_pixel_total : 49464 time to create 1 rle with old method : 0.057086944580078125 length of segment : 533 time for calcul the mask position with numpy : 0.0010225772857666016 nb_pixel_total : 28862 time to create 1 rle with old method : 0.03511667251586914 length of segment : 167 time for calcul the mask position with numpy : 0.00024771690368652344 nb_pixel_total : 3954 time to create 1 rle with old method : 0.0050046443939208984 length of segment : 97 time for calcul the mask position with numpy : 0.00020503997802734375 nb_pixel_total : 7094 time to create 1 rle with old method : 0.009574174880981445 length of segment : 98 time for calcul the mask position with numpy : 0.002930879592895508 nb_pixel_total : 93031 time to create 1 rle with old method : 0.10875058174133301 length of segment : 781 time for calcul the mask position with numpy : 0.0014829635620117188 nb_pixel_total : 62984 time to create 1 rle with old method : 0.08897519111633301 length of segment : 403 time spent for convertir_results : 18.361449718475342 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 176 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 60086 save missing photos in datou_result : time spend for datou_step_exec : 92.43112206459045 time spend to save output : 14.235987663269043 total time spend for step 1 : 106.6671097278595 step2:crop_condition Fri Mar 14 22:28:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 176 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 28 About to insert : list_path_to_insert length 28 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 47 About to insert : list_path_to_insert length 47 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1344296900, 1344273074, 1344273045, 1344273005, 1344272622, 1344272597, 1344272521, 1344272121, 1344272109, 1344272106] Looping around the photos to save general results len do output : 102 /-3713211397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922342Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922343Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713046578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3713211417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3712922399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344296900', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273074', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273045', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273005', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272622', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272597', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272521', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272121', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272109', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272106', None, None, None, None, None, '2671130') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 316 time used for this insertion : 0.036322832107543945 save_final save missing photos in datou_result : time spend for datou_step_exec : 50.11330032348633 time spend to save output : 0.039270877838134766 total time spend for step 2 : 50.15257120132446 step3:thcl Fri Mar 14 22:29:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.01197195053100586 time to convert the images to numpy array : 0.07254791259765625 time to import caffe and check if the image exist : 0.018088102340698242 time to convert the images to numpy array : 0.3671557903289795 time to import caffe and check if the image exist : 0.01689767837524414 time to convert the images to numpy array : 0.3683950901031494 time to import caffe and check if the image exist : 0.022289514541625977 time to convert the images to numpy array : 0.39177727699279785 time to import caffe and check if the image exist : 0.023204326629638672 time to convert the images to numpy array : 0.4095175266265869 time to import caffe and check if the image exist : 0.013374567031860352 time to convert the images to numpy array : 0.41895413398742676 time to import caffe and check if the image exist : 0.01700878143310547 time to convert the images to numpy array : 0.4188864231109619 time to import caffe and check if the image exist : 0.02010345458984375 time to convert the images to numpy array : 0.42591214179992676 time to import caffe and check if the image exist : 0.012660741806030273 time to convert the images to numpy array : 0.44471216201782227 time to import caffe and check if the image exist : 0.017281532287597656 time to convert the images to numpy array : 0.5413525104522705 total time to convert the images to numpy array : 0.7848992347717285 list photo_ids error: [] list photo_ids correct : [-3712922341, -3712922421, -3712922399, -3712922417, -3712922430, -3712922415, -3712922422, -3712922446, -3712922437, -3712922439, -3712922440, -3712922444, -3712922452, -3712922441, -3713211417, -3712922458, -3712922455, -3712922457, -3712922319, -3712922346, -3712922358, -3712922394, -3712922426, -3712922344, -3712922335, -3712922343, -3712922368, -3712922373, -3712922374, -3712922375, -3712922372, -3712922389, -3712922392, -3713211413, -3712922403, -3712922405, -3712922424, -3712922438, -3712922433, -3712922436, -3712922432, -3712922456, -3712922325, -3712922339, -3712922400, -3712922401, -3712922418, -3712922363, -3712922357, -3712922361, -3712922360, -3712922356, -3712922371, -3712922382, -3713211412, -3712922369, -3712922377, -3712922366, -3712922462, -3713211398, -3713211409, -3713211404, -3713211400, -3712922317, -3712922322, -3712922326, -3712922345, -3712922353, -3712922331, -3713211405, -3712922327, -3712922354, -3712922380, -3712922376, -3712922398, -3712922423, -3712922431, -3712922427, -3712922459, -3712922461, -3713211397, -3713211395, -3713211399, -3713211402, -3713211401, -3712922315, -3712922328, -3712922333, -3712922336, -3712922342, -3712922337, -3712922385, -3712922378, -3712922381, -3712922379, -3712922397, -3712922390, -3712922391, -3713211414, -3713046578, -3712922410, -3712922406] number of photos to traite : 102 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3890 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3671 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.3008840084075928 time used to do the prediction : 0.23126816749572754 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3669 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3669 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.345484972000122 time used to do the prediction : 0.19362211227416992 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.1219253540039062e-05 save missing photos in datou_result : time spend for datou_step_exec : 10.687894821166992 time spend to save output : 0.0007081031799316406 total time spend for step 3 : 10.688602924346924 step4:merge_mask_thcl_custom Fri Mar 14 22:29:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 176 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1344296900, 1344273074, 1344273045, 1344273005, 1344272622, 1344272597, 1344272521, 1344272121, 1344272109, 1344272106] Looping around the photos to save general results len do output : 10 /1344296900Didn't retrieve data .Didn't retrieve data . /1344273074Didn't retrieve data .Didn't retrieve data . /1344273045Didn't retrieve data .Didn't retrieve data . /1344273005Didn't retrieve data .Didn't retrieve data . /1344272622Didn't retrieve data .Didn't retrieve data . /1344272597Didn't retrieve data .Didn't retrieve data . /1344272521Didn't retrieve data .Didn't retrieve data . /1344272121Didn't retrieve data .Didn't retrieve data . /1344272109Didn't retrieve data .Didn't retrieve data . /1344272106Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344296900', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273074', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273045', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273005', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272622', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272597', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272521', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272121', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272109', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272106', None, None, None, None, None, '2671130') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01303863525390625 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.07259106636047363 time spend to save output : 0.01362752914428711 total time spend for step 4 : 0.08621859550476074 step5:rle_unique_nms_with_priority Fri Mar 14 22:29:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 9.067753553390503 time for calcul the mask position with numpy : 0.4455702304840088 nb_pixel_total : 6551265 time to create 1 rle with new method : 0.5614447593688965 time for calcul the mask position with numpy : 0.04155087471008301 nb_pixel_total : 22920 time to create 1 rle with old method : 0.025523900985717773 time for calcul the mask position with numpy : 0.0389094352722168 nb_pixel_total : 23043 time to create 1 rle with old method : 0.02825927734375 time for calcul the mask position with numpy : 0.03936266899108887 nb_pixel_total : 11119 time to create 1 rle with old method : 0.015659332275390625 time for calcul the mask position with numpy : 0.040871620178222656 nb_pixel_total : 21978 time to create 1 rle with old method : 0.027144193649291992 time for calcul the mask position with numpy : 0.03891420364379883 nb_pixel_total : 2423 time to create 1 rle with old method : 0.0029075145721435547 time for calcul the mask position with numpy : 0.04068303108215332 nb_pixel_total : 2218 time to create 1 rle with old method : 0.002628803253173828 time for calcul the mask position with numpy : 0.04068279266357422 nb_pixel_total : 13229 time to create 1 rle with old method : 0.01614093780517578 time for calcul the mask position with numpy : 0.04578828811645508 nb_pixel_total : 3981 time to create 1 rle with old method : 0.007107973098754883 time for calcul the mask position with numpy : 0.04094433784484863 nb_pixel_total : 4101 time to create 1 rle with old method : 0.005677938461303711 time for calcul the mask position with numpy : 0.03901839256286621 nb_pixel_total : 1027 time to create 1 rle with old method : 0.001264333724975586 time for calcul the mask position with numpy : 0.040660858154296875 nb_pixel_total : 46729 time to create 1 rle with old method : 0.05230283737182617 time for calcul the mask position with numpy : 0.03591322898864746 nb_pixel_total : 55841 time to create 1 rle with old method : 0.06526327133178711 time for calcul the mask position with numpy : 0.04696059226989746 nb_pixel_total : 3538 time to create 1 rle with old method : 0.005854129791259766 time for calcul the mask position with numpy : 0.04650616645812988 nb_pixel_total : 176961 time to create 1 rle with new method : 0.8318119049072266 time for calcul the mask position with numpy : 0.03217482566833496 nb_pixel_total : 51298 time to create 1 rle with old method : 0.05943608283996582 time for calcul the mask position with numpy : 0.0309598445892334 nb_pixel_total : 56797 time to create 1 rle with old method : 0.06609749794006348 time for calcul the mask position with numpy : 0.029101133346557617 nb_pixel_total : 1772 time to create 1 rle with old method : 0.002131938934326172 create new chi : 2.954803466796875 time to delete rle : 0.01404714584350586 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 9309 TO DO : save crop sub photo not yet done ! save time : 0.5543286800384521 nb_obj : 13 nb_hashtags : 5 time to prepare the origin masks : 9.29772138595581 time for calcul the mask position with numpy : 0.5553483963012695 nb_pixel_total : 6153803 time to create 1 rle with new method : 0.6011264324188232 time for calcul the mask position with numpy : 0.02397608757019043 nb_pixel_total : 87097 time to create 1 rle with old method : 0.10062289237976074 time for calcul the mask position with numpy : 0.020928621292114258 nb_pixel_total : 8431 time to create 1 rle with old method : 0.009450435638427734 time for calcul the mask position with numpy : 0.037174224853515625 nb_pixel_total : 274984 time to create 1 rle with new method : 1.0830719470977783 time for calcul the mask position with numpy : 0.03560471534729004 nb_pixel_total : 32427 time to create 1 rle with old method : 0.036377668380737305 time for calcul the mask position with numpy : 0.035710811614990234 nb_pixel_total : 10644 time to create 1 rle with old method : 0.011875629425048828 time for calcul the mask position with numpy : 0.03437638282775879 nb_pixel_total : 5521 time to create 1 rle with old method : 0.006208896636962891 time for calcul the mask position with numpy : 0.03490614891052246 nb_pixel_total : 905 time to create 1 rle with old method : 0.001043558120727539 time for calcul the mask position with numpy : 0.04229617118835449 nb_pixel_total : 357814 time to create 1 rle with new method : 0.45151615142822266 time for calcul the mask position with numpy : 0.035132646560668945 nb_pixel_total : 35238 time to create 1 rle with old method : 0.04088187217712402 time for calcul the mask position with numpy : 0.021733522415161133 nb_pixel_total : 14867 time to create 1 rle with old method : 0.01698589324951172 time for calcul the mask position with numpy : 0.021674633026123047 nb_pixel_total : 15796 time to create 1 rle with old method : 0.017743825912475586 time for calcul the mask position with numpy : 0.023829936981201172 nb_pixel_total : 44793 time to create 1 rle with old method : 0.05354881286621094 time for calcul the mask position with numpy : 0.022495508193969727 nb_pixel_total : 7920 time to create 1 rle with old method : 0.009770631790161133 create new chi : 3.4748787879943848 time to delete rle : 0.0014662742614746094 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 10045 TO DO : save crop sub photo not yet done ! save time : 1.2054469585418701 nb_obj : 26 nb_hashtags : 6 time to prepare the origin masks : 9.14491319656372 time for calcul the mask position with numpy : 0.6356816291809082 nb_pixel_total : 6037951 time to create 1 rle with new method : 0.6025106906890869 time for calcul the mask position with numpy : 0.028803110122680664 nb_pixel_total : 95003 time to create 1 rle with old method : 0.10468173027038574 time for calcul the mask position with numpy : 0.02782583236694336 nb_pixel_total : 2210 time to create 1 rle with old method : 0.002605915069580078 time for calcul the mask position with numpy : 0.027656078338623047 nb_pixel_total : 2700 time to create 1 rle with old method : 0.003042936325073242 time for calcul the mask position with numpy : 0.02765059471130371 nb_pixel_total : 1651 time to create 1 rle with old method : 0.0020792484283447266 time for calcul the mask position with numpy : 0.02868032455444336 nb_pixel_total : 31806 time to create 1 rle with old method : 0.03656721115112305 time for calcul the mask position with numpy : 0.029773473739624023 nb_pixel_total : 16641 time to create 1 rle with old method : 0.02316427230834961 time for calcul the mask position with numpy : 0.03026890754699707 nb_pixel_total : 8963 time to create 1 rle with old method : 0.010339975357055664 time for calcul the mask position with numpy : 0.02928328514099121 nb_pixel_total : 12486 time to create 1 rle with old method : 0.014069318771362305 time for calcul the mask position with numpy : 0.02918243408203125 nb_pixel_total : 94629 time to create 1 rle with old method : 0.10555791854858398 time for calcul the mask position with numpy : 0.028531789779663086 nb_pixel_total : 6586 time to create 1 rle with old method : 0.008003711700439453 time for calcul the mask position with numpy : 0.02938365936279297 nb_pixel_total : 11984 time to create 1 rle with old method : 0.014005422592163086 time for calcul the mask position with numpy : 0.029608726501464844 nb_pixel_total : 82501 time to create 1 rle with old method : 0.08947157859802246 time for calcul the mask position with numpy : 0.02814793586730957 nb_pixel_total : 28523 time to create 1 rle with old method : 0.032283782958984375 time for calcul the mask position with numpy : 0.02810525894165039 nb_pixel_total : 2916 time to create 1 rle with old method : 0.003385782241821289 time for calcul the mask position with numpy : 0.028648853302001953 nb_pixel_total : 24762 time to create 1 rle with old method : 0.027621030807495117 time for calcul the mask position with numpy : 0.029057025909423828 nb_pixel_total : 8 time to create 1 rle with old method : 5.173683166503906e-05 time for calcul the mask position with numpy : 0.029540300369262695 nb_pixel_total : 8576 time to create 1 rle with old method : 0.009783506393432617 time for calcul the mask position with numpy : 0.030649900436401367 nb_pixel_total : 149057 time to create 1 rle with old method : 0.19182658195495605 time for calcul the mask position with numpy : 0.028742313385009766 nb_pixel_total : 3575 time to create 1 rle with old method : 0.004635810852050781 time for calcul the mask position with numpy : 0.03245973587036133 nb_pixel_total : 2352 time to create 1 rle with old method : 0.004689455032348633 time for calcul the mask position with numpy : 0.029538393020629883 nb_pixel_total : 13522 time to create 1 rle with old method : 0.016080141067504883 time for calcul the mask position with numpy : 0.03304743766784668 nb_pixel_total : 299674 time to create 1 rle with new method : 0.6853640079498291 time for calcul the mask position with numpy : 0.02854156494140625 nb_pixel_total : 9979 time to create 1 rle with old method : 0.011534929275512695 time for calcul the mask position with numpy : 0.02879047393798828 nb_pixel_total : 98264 time to create 1 rle with old method : 0.11474990844726562 time for calcul the mask position with numpy : 0.02922677993774414 nb_pixel_total : 1837 time to create 1 rle with old method : 0.002490520477294922 time for calcul the mask position with numpy : 0.028829336166381836 nb_pixel_total : 2084 time to create 1 rle with old method : 0.0023987293243408203 create new chi : 3.57877254486084 time to delete rle : 0.0015916824340820312 batch 1 Loaded 27 chid ids of type : 3726 Number RLEs to save : 15117 TO DO : save crop sub photo not yet done ! save time : 3.658541202545166 nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 5.658982753753662 time for calcul the mask position with numpy : 0.4249258041381836 nb_pixel_total : 6476791 time to create 1 rle with new method : 0.8898983001708984 time for calcul the mask position with numpy : 0.02167654037475586 nb_pixel_total : 97663 time to create 1 rle with old method : 0.10843706130981445 time for calcul the mask position with numpy : 0.021817922592163086 nb_pixel_total : 27327 time to create 1 rle with old method : 0.03102850914001465 time for calcul the mask position with numpy : 0.023814916610717773 nb_pixel_total : 1888 time to create 1 rle with old method : 0.003232717514038086 time for calcul the mask position with numpy : 0.03628110885620117 nb_pixel_total : 155787 time to create 1 rle with new method : 0.47409677505493164 time for calcul the mask position with numpy : 0.03466463088989258 nb_pixel_total : 11530 time to create 1 rle with old method : 0.013319253921508789 time for calcul the mask position with numpy : 0.033919572830200195 nb_pixel_total : 14849 time to create 1 rle with old method : 0.016812801361083984 time for calcul the mask position with numpy : 0.03505992889404297 nb_pixel_total : 184639 time to create 1 rle with new method : 0.6152298450469971 time for calcul the mask position with numpy : 0.03304123878479004 nb_pixel_total : 79766 time to create 1 rle with old method : 0.08474421501159668 create new chi : 2.9861438274383545 time to delete rle : 0.0008006095886230469 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 7325 TO DO : save crop sub photo not yet done ! save time : 2.5202150344848633 nb_obj : 21 nb_hashtags : 8 time to prepare the origin masks : 17.877118587493896 time for calcul the mask position with numpy : 0.5401062965393066 nb_pixel_total : 6065153 time to create 1 rle with new method : 1.0734939575195312 time for calcul the mask position with numpy : 0.034364938735961914 nb_pixel_total : 47 time to create 1 rle with old method : 0.00019097328186035156 time for calcul the mask position with numpy : 0.03484368324279785 nb_pixel_total : 36497 time to create 1 rle with old method : 0.04156756401062012 time for calcul the mask position with numpy : 0.034850358963012695 nb_pixel_total : 8179 time to create 1 rle with old method : 0.009575366973876953 time for calcul the mask position with numpy : 0.03855299949645996 nb_pixel_total : 82355 time to create 1 rle with old method : 0.09203147888183594 time for calcul the mask position with numpy : 0.03902482986450195 nb_pixel_total : 248792 time to create 1 rle with new method : 0.5357959270477295 time for calcul the mask position with numpy : 0.03530097007751465 nb_pixel_total : 17849 time to create 1 rle with old method : 0.024500131607055664 time for calcul the mask position with numpy : 0.034255266189575195 nb_pixel_total : 20758 time to create 1 rle with old method : 0.023055076599121094 time for calcul the mask position with numpy : 0.034052371978759766 nb_pixel_total : 16366 time to create 1 rle with old method : 0.018553972244262695 time for calcul the mask position with numpy : 0.03530240058898926 nb_pixel_total : 80214 time to create 1 rle with old method : 0.09553885459899902 time for calcul the mask position with numpy : 0.039441823959350586 nb_pixel_total : 32529 time to create 1 rle with old method : 0.04713749885559082 time for calcul the mask position with numpy : 0.03629612922668457 nb_pixel_total : 15700 time to create 1 rle with old method : 0.025302410125732422 time for calcul the mask position with numpy : 0.044123172760009766 nb_pixel_total : 105892 time to create 1 rle with old method : 0.115081787109375 time for calcul the mask position with numpy : 0.03387856483459473 nb_pixel_total : 2737 time to create 1 rle with old method : 0.0031380653381347656 time for calcul the mask position with numpy : 0.034209489822387695 nb_pixel_total : 7402 time to create 1 rle with old method : 0.008629560470581055 time for calcul the mask position with numpy : 0.03518939018249512 nb_pixel_total : 70206 time to create 1 rle with old method : 0.08490133285522461 time for calcul the mask position with numpy : 0.03609156608581543 nb_pixel_total : 73040 time to create 1 rle with old method : 0.0818777084350586 time for calcul the mask position with numpy : 0.03374648094177246 nb_pixel_total : 8169 time to create 1 rle with old method : 0.010123014450073242 time for calcul the mask position with numpy : 0.036048173904418945 nb_pixel_total : 8103 time to create 1 rle with old method : 0.009290456771850586 time for calcul the mask position with numpy : 0.03428459167480469 nb_pixel_total : 19959 time to create 1 rle with old method : 0.023688316345214844 time for calcul the mask position with numpy : 0.035898447036743164 nb_pixel_total : 38553 time to create 1 rle with old method : 0.0444028377532959 time for calcul the mask position with numpy : 0.034631967544555664 nb_pixel_total : 91740 time to create 1 rle with old method : 0.10338401794433594 create new chi : 3.8335366249084473 time to delete rle : 0.0020411014556884766 batch 1 Loaded 24 chid ids of type : 3726 Number RLEs to save : 13698 TO DO : save crop sub photo not yet done ! save time : 4.462457895278931 nb_obj : 15 nb_hashtags : 7 time to prepare the origin masks : 12.479886293411255 time for calcul the mask position with numpy : 0.23786258697509766 nb_pixel_total : 4370719 time to create 1 rle with new method : 0.4237833023071289 time for calcul the mask position with numpy : 0.03931617736816406 nb_pixel_total : 3093 time to create 1 rle with old method : 0.0038213729858398438 time for calcul the mask position with numpy : 0.041248321533203125 nb_pixel_total : 47403 time to create 1 rle with old method : 0.054575443267822266 time for calcul the mask position with numpy : 0.046999216079711914 nb_pixel_total : 3758 time to create 1 rle with old method : 0.005434989929199219 time for calcul the mask position with numpy : 0.043111324310302734 nb_pixel_total : 2403 time to create 1 rle with old method : 0.0033926963806152344 time for calcul the mask position with numpy : 0.044007062911987305 nb_pixel_total : 139021 time to create 1 rle with old method : 0.15751886367797852 time for calcul the mask position with numpy : 0.03800797462463379 nb_pixel_total : 133229 time to create 1 rle with old method : 0.15992164611816406 time for calcul the mask position with numpy : 0.04621386528015137 nb_pixel_total : 286066 time to create 1 rle with new method : 0.33582329750061035 time for calcul the mask position with numpy : 0.026267051696777344 nb_pixel_total : 187069 time to create 1 rle with new method : 0.32970309257507324 time for calcul the mask position with numpy : 0.02448582649230957 nb_pixel_total : 147804 time to create 1 rle with old method : 0.17813920974731445 time for calcul the mask position with numpy : 0.024545907974243164 nb_pixel_total : 56013 time to create 1 rle with old method : 0.06350970268249512 time for calcul the mask position with numpy : 0.023870229721069336 nb_pixel_total : 100490 time to create 1 rle with old method : 0.12268257141113281 time for calcul the mask position with numpy : 0.028360843658447266 nb_pixel_total : 611 time to create 1 rle with old method : 0.0021407604217529297 time for calcul the mask position with numpy : 0.026556015014648438 nb_pixel_total : 47809 time to create 1 rle with old method : 0.07514286041259766 time for calcul the mask position with numpy : 0.03192305564880371 nb_pixel_total : 374058 time to create 1 rle with new method : 0.41949915885925293 time for calcul the mask position with numpy : 0.04025864601135254 nb_pixel_total : 1150694 time to create 1 rle with new method : 0.3136935234069824 create new chi : 3.5467875003814697 time to delete rle : 0.002574920654296875 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 20666 TO DO : save crop sub photo not yet done ! save time : 5.974732398986816 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 8.941839933395386 time for calcul the mask position with numpy : 0.49907684326171875 nb_pixel_total : 4439118 time to create 1 rle with new method : 1.2431159019470215 time for calcul the mask position with numpy : 0.044476985931396484 nb_pixel_total : 24842 time to create 1 rle with old method : 0.03135180473327637 time for calcul the mask position with numpy : 0.04790854454040527 nb_pixel_total : 357262 time to create 1 rle with new method : 0.7397670745849609 time for calcul the mask position with numpy : 0.05030226707458496 nb_pixel_total : 260288 time to create 1 rle with new method : 1.0366096496582031 time for calcul the mask position with numpy : 0.06373214721679688 nb_pixel_total : 72317 time to create 1 rle with old method : 0.08211207389831543 time for calcul the mask position with numpy : 0.05886983871459961 nb_pixel_total : 2658 time to create 1 rle with old method : 0.0038716793060302734 time for calcul the mask position with numpy : 0.06086230278015137 nb_pixel_total : 208148 time to create 1 rle with new method : 1.851992130279541 time for calcul the mask position with numpy : 0.05060386657714844 nb_pixel_total : 40302 time to create 1 rle with old method : 0.05028843879699707 time for calcul the mask position with numpy : 0.05848193168640137 nb_pixel_total : 2747 time to create 1 rle with old method : 0.004683732986450195 time for calcul the mask position with numpy : 0.0654606819152832 nb_pixel_total : 217092 time to create 1 rle with new method : 0.8421921730041504 time for calcul the mask position with numpy : 0.10054349899291992 nb_pixel_total : 1317716 time to create 1 rle with new method : 0.799424409866333 time for calcul the mask position with numpy : 0.05994415283203125 nb_pixel_total : 19553 time to create 1 rle with old method : 0.02570176124572754 time for calcul the mask position with numpy : 0.055837154388427734 nb_pixel_total : 88197 time to create 1 rle with old method : 0.10803079605102539 create new chi : 8.294366359710693 time to delete rle : 0.0034122467041015625 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 14362 TO DO : save crop sub photo not yet done ! save time : 3.0411221981048584 nb_obj : 17 nb_hashtags : 7 time to prepare the origin masks : 17.145772457122803 time for calcul the mask position with numpy : 0.5258071422576904 nb_pixel_total : 5193321 time to create 1 rle with new method : 0.7691593170166016 time for calcul the mask position with numpy : 0.03167891502380371 nb_pixel_total : 11064 time to create 1 rle with old method : 0.013268232345581055 time for calcul the mask position with numpy : 0.04024791717529297 nb_pixel_total : 116291 time to create 1 rle with old method : 0.169966459274292 time for calcul the mask position with numpy : 0.026587724685668945 nb_pixel_total : 47695 time to create 1 rle with old method : 0.05683302879333496 time for calcul the mask position with numpy : 0.03734135627746582 nb_pixel_total : 7703 time to create 1 rle with old method : 0.014609575271606445 time for calcul the mask position with numpy : 0.03332066535949707 nb_pixel_total : 176460 time to create 1 rle with new method : 0.6657922267913818 time for calcul the mask position with numpy : 0.028181791305541992 nb_pixel_total : 7370 time to create 1 rle with old method : 0.01104116439819336 time for calcul the mask position with numpy : 0.032428741455078125 nb_pixel_total : 2386 time to create 1 rle with old method : 0.0028624534606933594 time for calcul the mask position with numpy : 0.028397083282470703 nb_pixel_total : 10528 time to create 1 rle with old method : 0.012455463409423828 time for calcul the mask position with numpy : 0.03178262710571289 nb_pixel_total : 21520 time to create 1 rle with old method : 0.03007984161376953 time for calcul the mask position with numpy : 0.0304715633392334 nb_pixel_total : 64688 time to create 1 rle with old method : 0.08054494857788086 time for calcul the mask position with numpy : 0.03359174728393555 nb_pixel_total : 43063 time to create 1 rle with old method : 0.052384376525878906 time for calcul the mask position with numpy : 0.05044674873352051 nb_pixel_total : 819261 time to create 1 rle with new method : 0.7084147930145264 time for calcul the mask position with numpy : 0.0363311767578125 nb_pixel_total : 34770 time to create 1 rle with old method : 0.0407407283782959 time for calcul the mask position with numpy : 0.03363299369812012 nb_pixel_total : 53552 time to create 1 rle with old method : 0.06122612953186035 time for calcul the mask position with numpy : 0.033701181411743164 nb_pixel_total : 205332 time to create 1 rle with new method : 1.3297977447509766 time for calcul the mask position with numpy : 0.036643028259277344 nb_pixel_total : 89787 time to create 1 rle with old method : 0.11783647537231445 time for calcul the mask position with numpy : 0.035176992416381836 nb_pixel_total : 145449 time to create 1 rle with old method : 0.17249345779418945 create new chi : 5.569216251373291 time to delete rle : 0.002977609634399414 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 14605 TO DO : save crop sub photo not yet done ! save time : 2.109311580657959 nb_obj : 18 nb_hashtags : 5 time to prepare the origin masks : 12.086496353149414 time for calcul the mask position with numpy : 0.4403562545776367 nb_pixel_total : 5994942 time to create 1 rle with new method : 1.145738124847412 time for calcul the mask position with numpy : 0.05314517021179199 nb_pixel_total : 175491 time to create 1 rle with new method : 0.7982418537139893 time for calcul the mask position with numpy : 0.048926353454589844 nb_pixel_total : 64104 time to create 1 rle with old method : 0.5142152309417725 time for calcul the mask position with numpy : 0.05778145790100098 nb_pixel_total : 504 time to create 1 rle with old method : 0.0008876323699951172 time for calcul the mask position with numpy : 0.06077218055725098 nb_pixel_total : 185 time to create 1 rle with old method : 0.00030994415283203125 time for calcul the mask position with numpy : 0.07351374626159668 nb_pixel_total : 7005 time to create 1 rle with old method : 0.009314775466918945 time for calcul the mask position with numpy : 0.05387616157531738 nb_pixel_total : 55246 time to create 1 rle with old method : 0.06784772872924805 time for calcul the mask position with numpy : 0.0549623966217041 nb_pixel_total : 38802 time to create 1 rle with old method : 0.04745626449584961 time for calcul the mask position with numpy : 0.05585169792175293 nb_pixel_total : 113088 time to create 1 rle with old method : 0.14589619636535645 time for calcul the mask position with numpy : 0.049744367599487305 nb_pixel_total : 14314 time to create 1 rle with old method : 0.016586780548095703 time for calcul the mask position with numpy : 0.05183053016662598 nb_pixel_total : 3100 time to create 1 rle with old method : 0.0040285587310791016 time for calcul the mask position with numpy : 0.06583571434020996 nb_pixel_total : 268094 time to create 1 rle with new method : 0.8408830165863037 time for calcul the mask position with numpy : 0.039850711822509766 nb_pixel_total : 5968 time to create 1 rle with old method : 0.0072591304779052734 time for calcul the mask position with numpy : 0.05211210250854492 nb_pixel_total : 65490 time to create 1 rle with old method : 0.11762285232543945 time for calcul the mask position with numpy : 0.040259599685668945 nb_pixel_total : 44073 time to create 1 rle with old method : 0.059328556060791016 time for calcul the mask position with numpy : 0.050151824951171875 nb_pixel_total : 14822 time to create 1 rle with old method : 0.022891998291015625 time for calcul the mask position with numpy : 0.055437564849853516 nb_pixel_total : 21249 time to create 1 rle with old method : 0.02598261833190918 time for calcul the mask position with numpy : 0.05717611312866211 nb_pixel_total : 58374 time to create 1 rle with old method : 0.08698725700378418 time for calcul the mask position with numpy : 0.04894137382507324 nb_pixel_total : 105389 time to create 1 rle with old method : 0.12298846244812012 create new chi : 5.573433876037598 time to delete rle : 0.0024518966674804688 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 12336 TO DO : save crop sub photo not yet done ! save time : 6.915691614151001 nb_obj : 10 nb_hashtags : 6 time to prepare the origin masks : 10.005879878997803 time for calcul the mask position with numpy : 0.3609161376953125 nb_pixel_total : 6794170 time to create 1 rle with new method : 0.6962020397186279 time for calcul the mask position with numpy : 0.0262451171875 nb_pixel_total : 28685 time to create 1 rle with old method : 0.04017019271850586 time for calcul the mask position with numpy : 0.03601837158203125 nb_pixel_total : 3950 time to create 1 rle with old method : 0.0071523189544677734 time for calcul the mask position with numpy : 0.03196120262145996 nb_pixel_total : 34333 time to create 1 rle with old method : 0.06586194038391113 time for calcul the mask position with numpy : 0.02699565887451172 nb_pixel_total : 2540 time to create 1 rle with old method : 0.003099203109741211 time for calcul the mask position with numpy : 0.02790093421936035 nb_pixel_total : 1471 time to create 1 rle with old method : 0.0019931793212890625 time for calcul the mask position with numpy : 0.026017427444458008 nb_pixel_total : 64101 time to create 1 rle with old method : 0.07981753349304199 time for calcul the mask position with numpy : 0.03202033042907715 nb_pixel_total : 16664 time to create 1 rle with old method : 0.020992040634155273 time for calcul the mask position with numpy : 0.027544498443603516 nb_pixel_total : 3570 time to create 1 rle with old method : 0.004112720489501953 time for calcul the mask position with numpy : 0.027170896530151367 nb_pixel_total : 92545 time to create 1 rle with old method : 0.10211396217346191 time for calcul the mask position with numpy : 0.026051998138427734 nb_pixel_total : 8211 time to create 1 rle with old method : 0.009292364120483398 create new chi : 1.738152027130127 time to delete rle : 0.0009853839874267578 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 6544 TO DO : save crop sub photo not yet done ! save time : 0.7411725521087646 map_output_result : {1344296900: (0.0, 'Should be the crop_list due to order', 0), 1344273074: (0.0, 'Should be the crop_list due to order', 0), 1344273045: (0.0, 'Should be the crop_list due to order', 0), 1344273005: (0.0, 'Should be the crop_list due to order', 0), 1344272622: (0.0, 'Should be the crop_list due to order', 0), 1344272597: (0.0, 'Should be the crop_list due to order', 0), 1344272521: (0.0, 'Should be the crop_list due to order', 0), 1344272121: (0.0, 'Should be the crop_list due to order', 0), 1344272109: (0.0, 'Should be the crop_list due to order', 0), 1344272106: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1344296900, 1344273074, 1344273045, 1344273005, 1344272622, 1344272597, 1344272521, 1344272121, 1344272109, 1344272106] Looping around the photos to save general results len do output : 10 /1344296900.Didn't retrieve data . /1344273074.Didn't retrieve data . /1344273045.Didn't retrieve data . /1344273005.Didn't retrieve data . /1344272622.Didn't retrieve data . /1344272597.Didn't retrieve data . /1344272521.Didn't retrieve data . /1344272121.Didn't retrieve data . /1344272109.Didn't retrieve data . /1344272106.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344296900', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273074', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273045', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273005', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272622', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272597', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272521', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272121', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272109', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272106', None, None, None, None, None, '2671130') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014177560806274414 save_final save missing photos in datou_result : time spend for datou_step_exec : 187.21572351455688 time spend to save output : 0.2142620086669922 total time spend for step 5 : 187.42998552322388 step6:crop_condition Fri Mar 14 22:32:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 177 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741987951_1660800 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.6886394023895264 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741987958_1660800 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.7904725074768066 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741987964_1660800 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.069291830062866 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741987971_1660800 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3820726871490479 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741987975_1660800 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.2818658351898193 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741987978_1660800 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.7163052558898926 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741987981_1660800 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.6297726631164551 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1344296900, 1344273074, 1344273045, 1344273005, 1344272622, 1344272597, 1344272521, 1344272121, 1344272109, 1344272106] Looping around the photos to save general results len do output : 30 /1344332522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332553Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332566Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332567Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332570Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1344332572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344296900', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273074', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273045', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273005', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272622', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272597', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272521', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272121', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272109', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272106', None, None, None, None, None, '2671130') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 100 time used for this insertion : 0.03226470947265625 save_final save missing photos in datou_result : time spend for datou_step_exec : 41.34833788871765 time spend to save output : 0.06360578536987305 total time spend for step 6 : 41.411943674087524 step7:ventilate_hashtags_in_portfolio Fri Mar 14 22:33:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 21454786 get user id for portfolio 21454786 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21454786 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','environnement','autre_refus','flou','Carton_brun','plastique','mal_croppe','metal','kraft','Teint_Dans_La_Masse','papier','Carton_gris')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21454786 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','environnement','autre_refus','flou','Carton_brun','plastique','mal_croppe','metal','kraft','Teint_Dans_La_Masse','papier','Carton_gris')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21454786 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','environnement','autre_refus','flou','Carton_brun','plastique','mal_croppe','metal','kraft','Teint_Dans_La_Masse','papier','Carton_gris')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/21455397,21455398,21455400,21455401,21455402,21455403,21455404,21455405,21455406,21455407,21455408,21455409?tags=cartonnette,environnement,autre_refus,flou,Carton_brun,plastique,mal_croppe,metal,kraft,Teint_Dans_La_Masse,papier,Carton_gris Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1344296900, 1344273074, 1344273045, 1344273005, 1344272622, 1344272597, 1344272521, 1344272121, 1344272109, 1344272106] Looping around the photos to save general results len do output : 1 /21454786. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344296900', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273074', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273045', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273005', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272622', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272597', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272521', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272121', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272109', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272106', None, None, None, None, None, '2671130') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014781951904296875 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.3888437747955322 time spend to save output : 0.015180587768554688 total time spend for step 7 : 1.404024362564087 step8:final Fri Mar 14 22:33:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1344296900: ('0.07541344094702822',), 1344273074: ('0.07541344094702822',), 1344273045: ('0.07541344094702822',), 1344273005: ('0.07541344094702822',), 1344272622: ('0.07541344094702822',), 1344272597: ('0.07541344094702822',), 1344272521: ('0.07541344094702822',), 1344272121: ('0.07541344094702822',), 1344272109: ('0.07541344094702822',), 1344272106: ('0.07541344094702822',)} new output for save of step final : {1344296900: ('0.07541344094702822',), 1344273074: ('0.07541344094702822',), 1344273045: ('0.07541344094702822',), 1344273005: ('0.07541344094702822',), 1344272622: ('0.07541344094702822',), 1344272597: ('0.07541344094702822',), 1344272521: ('0.07541344094702822',), 1344272121: ('0.07541344094702822',), 1344272109: ('0.07541344094702822',), 1344272106: ('0.07541344094702822',)} [1344296900, 1344273074, 1344273045, 1344273005, 1344272622, 1344272597, 1344272521, 1344272121, 1344272109, 1344272106] Looping around the photos to save general results len do output : 10 /1344296900.Didn't retrieve data . /1344273074.Didn't retrieve data . /1344273045.Didn't retrieve data . /1344273005.Didn't retrieve data . /1344272622.Didn't retrieve data . /1344272597.Didn't retrieve data . /1344272521.Didn't retrieve data . /1344272121.Didn't retrieve data . /1344272109.Didn't retrieve data . /1344272106.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344296900', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273074', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273045', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273005', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272622', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272597', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272521', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272121', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272109', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272106', None, None, None, None, None, '2671130') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014683961868286133 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11332845687866211 time spend to save output : 0.015657424926757812 total time spend for step 8 : 0.12898588180541992 step9:velours_tree Fri Mar 14 22:33:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.311755895614624 time spend to save output : 5.125999450683594e-05 total time spend for step 9 : 0.31180715560913086 step10:send_mail_cod Fri Mar 14 22:33:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P21454786_14-03-2025_22_33_07.pdf 21455397 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette214553971741987987 21455400 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette214554001741987987 21455401 imagette214554011741987988 21455403 change filename to text .change filename to text .imagette214554031741987988 21455404 imagette214554041741987988 21455405 imagette214554051741987988 21455406 change filename to text .imagette214554061741987988 21455402 change filename to text .change filename to text .change filename to text .imagette214554021741987988 21455407 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette214554071741987988 21455409 change filename to text .change filename to text .change filename to text .change filename to text .imagette214554091741987989 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21454786 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21455397,21455398,21455400,21455401,21455402,21455403,21455404,21455405,21455406,21455407,21455408,21455409?tags=cartonnette,environnement,autre_refus,flou,Carton_brun,plastique,mal_croppe,metal,kraft,Teint_Dans_La_Masse,papier,Carton_gris your option no_mail is active, we will not send the real mail to your client args[1344296900] : ('0.07541344094702822',) no score found for photo 1344296900 We are sending mail with results at report@fotonower.com args[1344273074] : ('0.07541344094702822',) no score found for photo 1344273074 We are sending mail with results at report@fotonower.com args[1344273045] : ('0.07541344094702822',) no score found for photo 1344273045 We are sending mail with results at report@fotonower.com args[1344273005] : ('0.07541344094702822',) no score found for photo 1344273005 We are sending mail with results at report@fotonower.com args[1344272622] : ('0.07541344094702822',) no score found for photo 1344272622 We are sending mail with results at report@fotonower.com args[1344272597] : ('0.07541344094702822',) no score found for photo 1344272597 We are sending mail with results at report@fotonower.com args[1344272521] : ('0.07541344094702822',) no score found for photo 1344272521 We are sending mail with results at report@fotonower.com args[1344272121] : ('0.07541344094702822',) no score found for photo 1344272121 We are sending mail with results at report@fotonower.com args[1344272109] : ('0.07541344094702822',) no score found for photo 1344272109 We are sending mail with results at report@fotonower.com args[1344272106] : ('0.07541344094702822',) no score found for photo 1344272106 We are sending mail with results at report@fotonower.com refus_total : 0.07541344094702822 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21454786 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1344272099,1344272109,1344272121,1344272622,1344273005,1344273045,1344296900,1344272597,1344272106,1344272521,1344273074) Found this number of photos: 11 begin to download photo : 1344272099 begin to download photo : 1344272622 begin to download photo : 1344296900 begin to download photo : 1344272521 download finish for photo 1344272622 begin to download photo : 1344273005 download finish for photo 1344272521 begin to download photo : 1344273074 download finish for photo 1344296900 begin to download photo : 1344272597 download finish for photo 1344272099 begin to download photo : 1344272109 download finish for photo 1344272597 begin to download photo : 1344272106 download finish for photo 1344273074 download finish for photo 1344273005 begin to download photo : 1344273045 download finish for photo 1344272109 begin to download photo : 1344272121 download finish for photo 1344273045 download finish for photo 1344272121 download finish for photo 1344272106 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21454786_14-03-2025_22_33_07.pdf results_Auto_P21454786_14-03-2025_22_33_07.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21454786_14-03-2025_22_33_07.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','21454786','results_Auto_P21454786_14-03-2025_22_33_07.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21454786_14-03-2025_22_33_07.pdf','pdf','','0.56','0.07541344094702822') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1344296900, 1344273074, 1344273045, 1344273005, 1344272622, 1344272597, 1344272521, 1344272121, 1344272109, 1344272106] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344296900', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273074', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273045', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273005', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272622', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272597', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272521', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272121', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272109', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272106', None, None, None, None, None, '2671130') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.011979341506958008 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.885047674179077 time spend to save output : 0.012197732925415039 total time spend for step 10 : 4.897245407104492 step11:split_time_score Fri Mar 14 22:33:12 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('20', 11),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 14032025 21454786 Nombre de photos uploadées : 11 / 23040 (0%) 14032025 21454786 Nombre de photos taguées (types de déchets): 0 / 11 (0%) 14032025 21454786 Nombre de photos taguées (volume) : 0 / 11 (0%) elapsed_time : load_data_split_time_score 3.814697265625e-06 elapsed_time : order_list_meta_photo_and_scores 1.0728836059570312e-05 ??????????? elapsed_time : fill_and_build_computed_from_old_data 0.0011446475982666016 elapsed_time : insert_dashboard_record_day_entry 0.022139549255371094 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.05674420462057292 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21416179_14-03-2025_09_50_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21416179 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21416179 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454777 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454778 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454779 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454780 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454781 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425295_14-03-2025_12_57_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425295 order by id desc limit 1 Qualite : 0.06438740016588668 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425296_14-03-2025_22_27_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425296 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21425296 AND mptpi.`type`=4207 To do Qualite : 0.07493675983023386 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425297_14-03-2025_13_15_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425297 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21425297 AND mptpi.`type`=4207 To do Qualite : 0.06468981098604176 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425298_14-03-2025_13_00_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425298 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21425298 AND mptpi.`type`=4207 To do Qualite : 0.03991597164895486 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21418608_14-03-2025_10_22_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21418608 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21418608 AND mptpi.`type`=4207 To do Qualite : 0.04468489596863188 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21418609_14-03-2025_11_03_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21418609 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21418609 AND mptpi.`type`=4207 To do Qualite : 0.4241951561233941 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21418610_14-03-2025_10_57_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21418610 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21418610 AND mptpi.`type`=4200 To do Qualite : 0.04322752056069609 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21421445_14-03-2025_11_38_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21421445 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21421445 AND mptpi.`type`=4209 To do Qualite : 0.06154173459865928 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21421446_14-03-2025_11_42_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21421446 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21421446 AND mptpi.`type`=3726 To do Qualite : 0.10195782322821316 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425299_14-03-2025_15_56_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425299 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21425299 AND mptpi.`type`=3327 To do Qualite : 0.2448109360051482 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425300_14-03-2025_13_04_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425300 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21425300 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425301_14-03-2025_12_55_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425301 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425302_14-03-2025_13_04_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425302 order by id desc limit 1 Qualite : 0.2444303078098524 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425303_14-03-2025_13_03_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425303 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21425303 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21425304_14-03-2025_13_12_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21425304 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21432634_14-03-2025_15_31_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21432634 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21432635_14-03-2025_15_36_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21432635 order by id desc limit 1 Qualite : 0.09739104025080277 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21438130_14-03-2025_18_05_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21438130 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21438130 AND mptpi.`type`=4207 To do Qualite : 0.07090489552008594 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21431749_14-03-2025_18_53_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21431749 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21431749 AND mptpi.`type`=4207 To do Qualite : 0.06705160025197379 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21432673_14-03-2025_19_17_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21432673 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21432673 AND mptpi.`type`=4207 To do Qualite : 0.10461227809355336 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21432698_14-03-2025_19_39_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21432698 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11512 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11521 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11520 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11516 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11516 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11519 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11519 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11513 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11513 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11577 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11577 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11515 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11514 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11514 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11523 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11515 doesn't seem to be define in the database( WARNING : type of input 3 of step 11514 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11512 doesn't seem to be define in the database( WARNING : type of input 2 of step 11516 doesn't seem to be define in the database( WARNING : output 1 of step 11512 have datatype=2 whereas input 1 of step 11519 have datatype=7 WARNING : type of output 2 of step 11519 doesn't seem to be define in the database( WARNING : type of input 1 of step 11513 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11515 have datatype=10 whereas input 3 of step 11522 have datatype=6 WARNING : type of input 2 of step 11577 doesn't seem to be define in the database( WARNING : output 1 of step 11513 have datatype=7 whereas input 2 of step 11577 have datatype=None WARNING : type of output 3 of step 11577 doesn't seem to be define in the database( WARNING : type of input 1 of step 11515 doesn't seem to be define in the database( WARNING : output 0 of step 11515 have datatype=10 whereas input 0 of step 11583 have datatype=18 WARNING : type of input 5 of step 11522 doesn't seem to be define in the database( WARNING : output 0 of step 11583 have datatype=11 whereas input 5 of step 11522 have datatype=None WARNING : type of output 1 of step 11521 doesn't seem to be define in the database( WARNING : type of input 3 of step 11516 doesn't seem to be define in the database( WARNING : type of output 1 of step 11520 doesn't seem to be define in the database( WARNING : type of input 3 of step 11516 doesn't seem to be define in the database( WARNING : output 0 of step 11519 have datatype=1 whereas input 0 of step 11513 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21432698 AND mptpi.`type`=4203 To do Qualite : 0.07836305391767237 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21438143_14-03-2025_21_36_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21438143 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21438143 AND mptpi.`type`=4211 To do Qualite : 0.20401060814950975 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21438144_14-03-2025_17_22_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21438144 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21438144 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21438145_14-03-2025_16_56_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21438145 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21438146_14-03-2025_17_04_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21438146 order by id desc limit 1 Qualite : 0.30039548642848996 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21438165_14-03-2025_17_35_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21438165 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21438165 AND mptpi.`type`=4200 To do Qualite : 0.19966875241126544 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21438172_14-03-2025_18_24_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21438172 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21438172 AND mptpi.`type`=3594 To do Qualite : 0.1254156099205101 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21438222_14-03-2025_17_46_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21438222 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21438222 AND mptpi.`type`=3327 To do Qualite : 0.05721174892922295 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21442635_14-03-2025_19_22_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21442635 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21442635 AND mptpi.`type`=4207 To do Qualite : 0.03860630360390389 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21442636_14-03-2025_19_34_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21442636 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21442636 AND mptpi.`type`=4207 To do Qualite : 0.07692270377880141 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21442639_14-03-2025_18_47_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21442639 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21442639 AND mptpi.`type`=4207 To do Qualite : 0.05919347915077416 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21442651_14-03-2025_18_33_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21442651 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21442651 AND mptpi.`type`=3726 To do Qualite : 0.05784947678062557 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21442653_14-03-2025_18_40_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21442653 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21442653 AND mptpi.`type`=4209 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21445934_14-03-2025_19_32_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21445934 order by id desc limit 1 Qualite : 0.21314390715777054 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21445935_14-03-2025_19_31_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21445935 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21445935 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21449624_14-03-2025_20_47_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21449624 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21449625 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21449626_14-03-2025_20_34_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21449626 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454783 order by id desc limit 1 Qualite : 0.07588156423407197 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21449628_14-03-2025_21_05_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21449628 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21449628 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454784 order by id desc limit 1 Qualite : 0.07225611408555906 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21454785_14-03-2025_22_32_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454785 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21454785 AND mptpi.`type`=4209 To do Qualite : 0.07541344094702822 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21454786_14-03-2025_22_33_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454786 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21454786 AND mptpi.`type`=3726 To do Qualite : 0.0895007319068991 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21454787_14-03-2025_22_28_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454787 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21454787 AND mptpi.`type`=4207 To do Qualite : 0.06004678767204481 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21454788_14-03-2025_22_26_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21454788 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21454788 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'14032025': {'nb_upload': 11, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1344296900, 1344273074, 1344273045, 1344273005, 1344272622, 1344272597, 1344272521, 1344272121, 1344272109, 1344272106] Looping around the photos to save general results len do output : 1 /21454786Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344296900', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273074', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273045', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344273005', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272622', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272597', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272521', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272121', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272109', None, None, None, None, None, '2671130') ('3459', None, None, None, None, None, None, None, '2671130') ('3459', '21454786', '1344272106', None, None, None, None, None, '2671130') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014198064804077148 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.568742752075195 time spend to save output : 0.014578580856323242 total time spend for step 11 : 14.583321332931519 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 180.46user 153.90system 7:03.33elapsed 78%CPU (0avgtext+0avgdata 4433496maxresident)k 1203744inputs+109160outputs (21598major+14282108minor)pagefaults 0swaps