python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1798213 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 20>10 please execute split_portfolio.py -i 21247982 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2656120'] with mtr_portfolio_ids : ['21247982'] and first list_photo_ids : [] new path : /proc/1798213/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.5295729637145996 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Mar 11 00:46:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 3000 l 3632 free memory gpu now : 2320 wait 20 seconds l 3637 free memory gpu now : 2320 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-11 00:46:53.327088: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-11 00:46:53.359157: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-11 00:46:53.361938: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f173c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-11 00:46:53.361972: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-11 00:46:53.367578: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-11 00:46:53.671369: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x181bef90 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-11 00:46:53.671436: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-11 00:46:53.672926: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 00:46:53.675166: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 00:46:53.711377: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 00:46:53.732822: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 00:46:53.737038: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 00:46:53.772072: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 00:46:53.778116: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 00:46:53.843517: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 00:46:53.845608: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 00:46:53.846257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 00:46:53.847944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 00:46:53.847984: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 00:46:53.848003: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 00:46:53.850123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3061 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-11 00:46:54.439528: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 00:46:54.439727: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 00:46:54.439748: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 00:46:54.439765: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 00:46:54.439781: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 00:46:54.439798: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 00:46:54.439812: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 00:46:54.439827: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 00:46:54.440788: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 00:46:54.441808: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 00:46:54.441878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 00:46:54.441900: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 00:46:54.441917: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 00:46:54.441935: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 00:46:54.441951: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 00:46:54.441966: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 00:46:54.441984: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 00:46:54.442963: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 00:46:54.443002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 00:46:54.443012: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 00:46:54.443020: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 00:46:54.444023: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3061 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-11 00:47:05.490459: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 00:47:05.876420: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 00:47:07.951414: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:07.952274: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.24G (2407307520 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:07.952309: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:07.953478: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:07.953503: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:07.960848: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:07.960879: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:07.961520: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:07.961538: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:07.968227: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:07.968252: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:07.968904: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:07.968921: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:08.003584: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.003672: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:08.004539: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.004561: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:08.011117: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.011196: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:08.011975: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.012001: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-03-11 00:47:08.047282: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.047986: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.049877: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.050518: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.092434: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.093131: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.095429: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.096077: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.125997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.126725: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.129098: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.130141: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.139225: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.139935: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.141898: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.142542: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.149354: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.150058: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.151889: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.152544: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.184361: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.185052: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.185688: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.186318: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.191520: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.192221: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.212952: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.213662: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.214304: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.214974: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.231291: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.232001: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.232640: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.233275: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.238588: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.239344: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.244669: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.245392: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.258535: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.259271: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.263694: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.264366: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.265004: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.265645: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.266787: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.267483: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.278883: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.279624: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.280281: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.280907: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.281536: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.282167: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.298381: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.299147: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.356951: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.357648: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.358316: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.358987: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.367344: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.368022: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.394448: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.395251: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.395900: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.396524: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.400785: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.401441: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.402073: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.402695: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.405117: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.405157: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-03-11 00:47:08.414237: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.415117: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.425670: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.426744: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.427875: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.428987: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.430102: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-03-11 00:47:08.431136: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.49G (2674786304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 Detection mask done ! Trying to reset tf kernel 1800405 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2320 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3513 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0017216205596923828 nb_pixel_total : 60019 time to create 1 rle with old method : 0.07077169418334961 length of segment : 164 time for calcul the mask position with numpy : 0.016451358795166016 nb_pixel_total : 387324 time to create 1 rle with new method : 0.1910548210144043 length of segment : 770 time for calcul the mask position with numpy : 0.003894805908203125 nb_pixel_total : 218986 time to create 1 rle with new method : 0.0177919864654541 length of segment : 735 time for calcul the mask position with numpy : 0.0015575885772705078 nb_pixel_total : 69572 time to create 1 rle with old method : 0.07841920852661133 length of segment : 832 time for calcul the mask position with numpy : 0.0030918121337890625 nb_pixel_total : 151362 time to create 1 rle with new method : 0.01203465461730957 length of segment : 519 time for calcul the mask position with numpy : 0.002048969268798828 nb_pixel_total : 108660 time to create 1 rle with old method : 0.12090945243835449 length of segment : 471 time for calcul the mask position with numpy : 0.0004963874816894531 nb_pixel_total : 17678 time to create 1 rle with old method : 0.020272493362426758 length of segment : 101 time for calcul the mask position with numpy : 0.00019049644470214844 nb_pixel_total : 5166 time to create 1 rle with old method : 0.006227016448974609 length of segment : 120 time for calcul the mask position with numpy : 0.0006906986236572266 nb_pixel_total : 25756 time to create 1 rle with old method : 0.031241893768310547 length of segment : 209 time for calcul the mask position with numpy : 0.004735469818115234 nb_pixel_total : 152979 time to create 1 rle with new method : 0.0199129581451416 length of segment : 474 time for calcul the mask position with numpy : 0.0011904239654541016 nb_pixel_total : 49462 time to create 1 rle with old method : 0.05977439880371094 length of segment : 285 time for calcul the mask position with numpy : 0.0005936622619628906 nb_pixel_total : 26612 time to create 1 rle with old method : 0.03753018379211426 length of segment : 148 time for calcul the mask position with numpy : 0.0046732425689697266 nb_pixel_total : 219572 time to create 1 rle with new method : 0.013601064682006836 length of segment : 415 time for calcul the mask position with numpy : 0.0027396678924560547 nb_pixel_total : 138902 time to create 1 rle with old method : 0.16777467727661133 length of segment : 491 time for calcul the mask position with numpy : 0.003443002700805664 nb_pixel_total : 176948 time to create 1 rle with new method : 0.01526784896850586 length of segment : 506 time for calcul the mask position with numpy : 0.0018973350524902344 nb_pixel_total : 122429 time to create 1 rle with old method : 0.14360642433166504 length of segment : 317 time for calcul the mask position with numpy : 0.0004944801330566406 nb_pixel_total : 18709 time to create 1 rle with old method : 0.02240920066833496 length of segment : 208 time for calcul the mask position with numpy : 0.002485036849975586 nb_pixel_total : 112682 time to create 1 rle with old method : 0.1529703140258789 length of segment : 496 time for calcul the mask position with numpy : 0.0002288818359375 nb_pixel_total : 3594 time to create 1 rle with old method : 0.006298065185546875 length of segment : 115 time for calcul the mask position with numpy : 0.0027294158935546875 nb_pixel_total : 116208 time to create 1 rle with old method : 0.21207785606384277 length of segment : 461 time for calcul the mask position with numpy : 0.004610300064086914 nb_pixel_total : 14165 time to create 1 rle with old method : 0.018082618713378906 length of segment : 172 time for calcul the mask position with numpy : 0.0022134780883789062 nb_pixel_total : 154718 time to create 1 rle with new method : 0.005015373229980469 length of segment : 619 time for calcul the mask position with numpy : 0.25760841369628906 nb_pixel_total : 2881889 time to create 1 rle with new method : 0.5049655437469482 length of segment : 2371 time for calcul the mask position with numpy : 0.003766298294067383 nb_pixel_total : 81574 time to create 1 rle with old method : 0.09771275520324707 length of segment : 577 time for calcul the mask position with numpy : 0.004046201705932617 nb_pixel_total : 168184 time to create 1 rle with new method : 0.01285696029663086 length of segment : 406 time for calcul the mask position with numpy : 0.00042366981506347656 nb_pixel_total : 15042 time to create 1 rle with old method : 0.0179445743560791 length of segment : 144 time for calcul the mask position with numpy : 0.0022735595703125 nb_pixel_total : 138673 time to create 1 rle with old method : 0.15908479690551758 length of segment : 656 time for calcul the mask position with numpy : 0.004047870635986328 nb_pixel_total : 137782 time to create 1 rle with old method : 0.15828418731689453 length of segment : 489 time for calcul the mask position with numpy : 0.0006430149078369141 nb_pixel_total : 16575 time to create 1 rle with old method : 0.019814252853393555 length of segment : 243 time for calcul the mask position with numpy : 0.0003581047058105469 nb_pixel_total : 3910 time to create 1 rle with old method : 0.006762266159057617 length of segment : 79 time for calcul the mask position with numpy : 0.002565622329711914 nb_pixel_total : 114854 time to create 1 rle with old method : 0.1308600902557373 length of segment : 499 time for calcul the mask position with numpy : 0.00043845176696777344 nb_pixel_total : 7283 time to create 1 rle with old method : 0.008866071701049805 length of segment : 167 time for calcul the mask position with numpy : 0.0011286735534667969 nb_pixel_total : 30777 time to create 1 rle with old method : 0.03762102127075195 length of segment : 249 time for calcul the mask position with numpy : 0.0023720264434814453 nb_pixel_total : 73954 time to create 1 rle with old method : 0.08604240417480469 length of segment : 324 time for calcul the mask position with numpy : 0.0015010833740234375 nb_pixel_total : 51775 time to create 1 rle with old method : 0.06108546257019043 length of segment : 310 time for calcul the mask position with numpy : 0.0016739368438720703 nb_pixel_total : 26864 time to create 1 rle with old method : 0.03313732147216797 length of segment : 330 time for calcul the mask position with numpy : 0.005741596221923828 nb_pixel_total : 194481 time to create 1 rle with new method : 0.00908350944519043 length of segment : 593 time for calcul the mask position with numpy : 0.0006072521209716797 nb_pixel_total : 9051 time to create 1 rle with old method : 0.010696172714233398 length of segment : 155 time for calcul the mask position with numpy : 0.0009160041809082031 nb_pixel_total : 30394 time to create 1 rle with old method : 0.039869070053100586 length of segment : 203 time for calcul the mask position with numpy : 0.000682830810546875 nb_pixel_total : 17383 time to create 1 rle with old method : 0.020955324172973633 length of segment : 127 time for calcul the mask position with numpy : 0.0010249614715576172 nb_pixel_total : 27003 time to create 1 rle with old method : 0.03291177749633789 length of segment : 330 time for calcul the mask position with numpy : 0.0005729198455810547 nb_pixel_total : 7383 time to create 1 rle with old method : 0.009032487869262695 length of segment : 186 time for calcul the mask position with numpy : 0.0047457218170166016 nb_pixel_total : 171638 time to create 1 rle with new method : 0.007928848266601562 length of segment : 533 time for calcul the mask position with numpy : 0.009196281433105469 nb_pixel_total : 265834 time to create 1 rle with new method : 0.03188920021057129 length of segment : 748 time for calcul the mask position with numpy : 0.002977132797241211 nb_pixel_total : 166300 time to create 1 rle with new method : 0.007333993911743164 length of segment : 493 time for calcul the mask position with numpy : 0.00875091552734375 nb_pixel_total : 226382 time to create 1 rle with new method : 0.01354360580444336 length of segment : 524 time for calcul the mask position with numpy : 0.0004894733428955078 nb_pixel_total : 12135 time to create 1 rle with old method : 0.015020370483398438 length of segment : 133 time for calcul the mask position with numpy : 0.0062999725341796875 nb_pixel_total : 203027 time to create 1 rle with new method : 0.009063243865966797 length of segment : 458 time for calcul the mask position with numpy : 0.0023925304412841797 nb_pixel_total : 75828 time to create 1 rle with old method : 0.09959220886230469 length of segment : 481 time for calcul the mask position with numpy : 0.00013256072998046875 nb_pixel_total : 1143 time to create 1 rle with old method : 0.0014653205871582031 length of segment : 65 time for calcul the mask position with numpy : 0.002933979034423828 nb_pixel_total : 94643 time to create 1 rle with old method : 0.11511516571044922 length of segment : 676 time for calcul the mask position with numpy : 0.006292819976806641 nb_pixel_total : 269755 time to create 1 rle with new method : 0.010795831680297852 length of segment : 714 time for calcul the mask position with numpy : 0.001773834228515625 nb_pixel_total : 29241 time to create 1 rle with old method : 0.03454947471618652 length of segment : 419 time for calcul the mask position with numpy : 0.006276369094848633 nb_pixel_total : 253342 time to create 1 rle with new method : 0.010781288146972656 length of segment : 972 time for calcul the mask position with numpy : 0.0006124973297119141 nb_pixel_total : 23084 time to create 1 rle with old method : 0.028333663940429688 length of segment : 177 time for calcul the mask position with numpy : 0.006643533706665039 nb_pixel_total : 271806 time to create 1 rle with new method : 0.01649928092956543 length of segment : 885 time for calcul the mask position with numpy : 0.0007035732269287109 nb_pixel_total : 24401 time to create 1 rle with old method : 0.029692649841308594 length of segment : 177 time for calcul the mask position with numpy : 0.004362344741821289 nb_pixel_total : 256731 time to create 1 rle with new method : 0.010066986083984375 length of segment : 872 time for calcul the mask position with numpy : 0.00022220611572265625 nb_pixel_total : 2159 time to create 1 rle with old method : 0.0028367042541503906 length of segment : 52 time for calcul the mask position with numpy : 0.002592802047729492 nb_pixel_total : 78945 time to create 1 rle with old method : 0.0924828052520752 length of segment : 518 time for calcul the mask position with numpy : 0.00014472007751464844 nb_pixel_total : 3418 time to create 1 rle with old method : 0.004123687744140625 length of segment : 62 time for calcul the mask position with numpy : 0.00018787384033203125 nb_pixel_total : 8894 time to create 1 rle with old method : 0.010428190231323242 length of segment : 144 time for calcul the mask position with numpy : 0.0007004737854003906 nb_pixel_total : 21307 time to create 1 rle with old method : 0.024457931518554688 length of segment : 195 time for calcul the mask position with numpy : 0.0025899410247802734 nb_pixel_total : 125257 time to create 1 rle with old method : 0.14303350448608398 length of segment : 439 time for calcul the mask position with numpy : 0.0002033710479736328 nb_pixel_total : 5119 time to create 1 rle with old method : 0.006611347198486328 length of segment : 71 time for calcul the mask position with numpy : 0.006899356842041016 nb_pixel_total : 300970 time to create 1 rle with new method : 0.014914751052856445 length of segment : 865 time for calcul the mask position with numpy : 0.001779317855834961 nb_pixel_total : 53097 time to create 1 rle with old method : 0.06365418434143066 length of segment : 374 time for calcul the mask position with numpy : 0.0018892288208007812 nb_pixel_total : 72655 time to create 1 rle with old method : 0.08349609375 length of segment : 320 time for calcul the mask position with numpy : 0.0011684894561767578 nb_pixel_total : 58767 time to create 1 rle with old method : 0.06689858436584473 length of segment : 400 time for calcul the mask position with numpy : 0.0007121562957763672 nb_pixel_total : 22851 time to create 1 rle with old method : 0.027030229568481445 length of segment : 186 time for calcul the mask position with numpy : 0.00016546249389648438 nb_pixel_total : 2359 time to create 1 rle with old method : 0.0029969215393066406 length of segment : 61 time for calcul the mask position with numpy : 0.00012135505676269531 nb_pixel_total : 1056 time to create 1 rle with old method : 0.0013349056243896484 length of segment : 59 time for calcul the mask position with numpy : 0.0016086101531982422 nb_pixel_total : 53579 time to create 1 rle with old method : 0.06197404861450195 length of segment : 422 time for calcul the mask position with numpy : 0.0076215267181396484 nb_pixel_total : 282960 time to create 1 rle with new method : 0.01848316192626953 length of segment : 994 time for calcul the mask position with numpy : 0.0010671615600585938 nb_pixel_total : 5219 time to create 1 rle with old method : 0.007017850875854492 length of segment : 287 time for calcul the mask position with numpy : 0.0001537799835205078 nb_pixel_total : 1661 time to create 1 rle with old method : 0.0020771026611328125 length of segment : 53 time for calcul the mask position with numpy : 0.0006849765777587891 nb_pixel_total : 14825 time to create 1 rle with old method : 0.017194271087646484 length of segment : 214 time for calcul the mask position with numpy : 0.009122133255004883 nb_pixel_total : 362217 time to create 1 rle with new method : 0.016289234161376953 length of segment : 793 time for calcul the mask position with numpy : 0.013726949691772461 nb_pixel_total : 485325 time to create 1 rle with new method : 0.3221280574798584 length of segment : 1036 time for calcul the mask position with numpy : 0.0002295970916748047 nb_pixel_total : 3337 time to create 1 rle with old method : 0.00432896614074707 length of segment : 77 time for calcul the mask position with numpy : 0.00013327598571777344 nb_pixel_total : 2742 time to create 1 rle with old method : 0.0033807754516601562 length of segment : 56 time for calcul the mask position with numpy : 0.018865346908569336 nb_pixel_total : 604836 time to create 1 rle with new method : 0.04799294471740723 length of segment : 1157 time for calcul the mask position with numpy : 0.0004601478576660156 nb_pixel_total : 18495 time to create 1 rle with old method : 0.02285909652709961 length of segment : 105 time for calcul the mask position with numpy : 0.00048804283142089844 nb_pixel_total : 21408 time to create 1 rle with old method : 0.024541616439819336 length of segment : 313 time for calcul the mask position with numpy : 0.0003132820129394531 nb_pixel_total : 5079 time to create 1 rle with old method : 0.006186723709106445 length of segment : 90 time for calcul the mask position with numpy : 0.0028104782104492188 nb_pixel_total : 167858 time to create 1 rle with new method : 0.004851579666137695 length of segment : 448 time for calcul the mask position with numpy : 0.0022690296173095703 nb_pixel_total : 81972 time to create 1 rle with old method : 0.09555220603942871 length of segment : 368 time for calcul the mask position with numpy : 0.00042176246643066406 nb_pixel_total : 11506 time to create 1 rle with old method : 0.014301061630249023 length of segment : 163 time for calcul the mask position with numpy : 0.0003554821014404297 nb_pixel_total : 5380 time to create 1 rle with old method : 0.006512165069580078 length of segment : 101 time for calcul the mask position with numpy : 0.0008320808410644531 nb_pixel_total : 22777 time to create 1 rle with old method : 0.0270388126373291 length of segment : 219 time for calcul the mask position with numpy : 0.002359628677368164 nb_pixel_total : 81401 time to create 1 rle with old method : 0.09549570083618164 length of segment : 309 time for calcul the mask position with numpy : 0.0001468658447265625 nb_pixel_total : 1071 time to create 1 rle with old method : 0.001455068588256836 length of segment : 43 time for calcul the mask position with numpy : 0.001111745834350586 nb_pixel_total : 14539 time to create 1 rle with old method : 0.018230199813842773 length of segment : 221 time for calcul the mask position with numpy : 0.0009021759033203125 nb_pixel_total : 26382 time to create 1 rle with old method : 0.03190469741821289 length of segment : 191 time for calcul the mask position with numpy : 0.0009894371032714844 nb_pixel_total : 23504 time to create 1 rle with old method : 0.02835822105407715 length of segment : 209 time for calcul the mask position with numpy : 0.00029850006103515625 nb_pixel_total : 3111 time to create 1 rle with old method : 0.006494045257568359 length of segment : 65 time for calcul the mask position with numpy : 0.0004017353057861328 nb_pixel_total : 5004 time to create 1 rle with old method : 0.006913661956787109 length of segment : 68 time for calcul the mask position with numpy : 0.00327301025390625 nb_pixel_total : 46603 time to create 1 rle with old method : 0.05820798873901367 length of segment : 485 time for calcul the mask position with numpy : 0.0026602745056152344 nb_pixel_total : 55559 time to create 1 rle with old method : 0.08064031600952148 length of segment : 317 time for calcul the mask position with numpy : 0.0007033348083496094 nb_pixel_total : 12302 time to create 1 rle with old method : 0.014706611633300781 length of segment : 172 time for calcul the mask position with numpy : 0.00017762184143066406 nb_pixel_total : 3568 time to create 1 rle with old method : 0.004555225372314453 length of segment : 63 time for calcul the mask position with numpy : 0.0003437995910644531 nb_pixel_total : 5246 time to create 1 rle with old method : 0.006289482116699219 length of segment : 124 time for calcul the mask position with numpy : 0.008697271347045898 nb_pixel_total : 284607 time to create 1 rle with new method : 0.03157162666320801 length of segment : 730 time for calcul the mask position with numpy : 0.00036025047302246094 nb_pixel_total : 5557 time to create 1 rle with old method : 0.006906986236572266 length of segment : 73 time for calcul the mask position with numpy : 0.0002238750457763672 nb_pixel_total : 3426 time to create 1 rle with old method : 0.0043256282806396484 length of segment : 68 time for calcul the mask position with numpy : 0.000152587890625 nb_pixel_total : 1610 time to create 1 rle with old method : 0.002081155776977539 length of segment : 52 time for calcul the mask position with numpy : 0.0006206035614013672 nb_pixel_total : 8509 time to create 1 rle with old method : 0.010200977325439453 length of segment : 168 time for calcul the mask position with numpy : 0.0024771690368652344 nb_pixel_total : 59431 time to create 1 rle with old method : 0.07607007026672363 length of segment : 319 time for calcul the mask position with numpy : 0.0022878646850585938 nb_pixel_total : 38740 time to create 1 rle with old method : 0.04566597938537598 length of segment : 240 time for calcul the mask position with numpy : 0.008285760879516602 nb_pixel_total : 296579 time to create 1 rle with new method : 0.015260934829711914 length of segment : 827 time for calcul the mask position with numpy : 0.00044035911560058594 nb_pixel_total : 5021 time to create 1 rle with old method : 0.006017923355102539 length of segment : 161 time for calcul the mask position with numpy : 0.00260162353515625 nb_pixel_total : 67444 time to create 1 rle with old method : 0.07971000671386719 length of segment : 373 time for calcul the mask position with numpy : 0.0013041496276855469 nb_pixel_total : 17591 time to create 1 rle with old method : 0.02532649040222168 length of segment : 197 time for calcul the mask position with numpy : 0.0010733604431152344 nb_pixel_total : 16841 time to create 1 rle with old method : 0.020174741744995117 length of segment : 138 time for calcul the mask position with numpy : 0.005956888198852539 nb_pixel_total : 168361 time to create 1 rle with new method : 0.006914854049682617 length of segment : 613 time for calcul the mask position with numpy : 0.00017952919006347656 nb_pixel_total : 2073 time to create 1 rle with old method : 0.00264739990234375 length of segment : 63 time for calcul the mask position with numpy : 0.0015249252319335938 nb_pixel_total : 24961 time to create 1 rle with old method : 0.03399252891540527 length of segment : 311 time for calcul the mask position with numpy : 0.0025022029876708984 nb_pixel_total : 109188 time to create 1 rle with old method : 0.1313023567199707 length of segment : 390 time for calcul the mask position with numpy : 0.0004837512969970703 nb_pixel_total : 3392 time to create 1 rle with old method : 0.004818439483642578 length of segment : 62 time for calcul the mask position with numpy : 0.003414154052734375 nb_pixel_total : 95065 time to create 1 rle with old method : 0.10893869400024414 length of segment : 431 time for calcul the mask position with numpy : 0.0006721019744873047 nb_pixel_total : 15293 time to create 1 rle with old method : 0.018347501754760742 length of segment : 300 time for calcul the mask position with numpy : 0.002875089645385742 nb_pixel_total : 102161 time to create 1 rle with old method : 0.13019919395446777 length of segment : 354 time for calcul the mask position with numpy : 0.001285552978515625 nb_pixel_total : 32701 time to create 1 rle with old method : 0.04290437698364258 length of segment : 258 time for calcul the mask position with numpy : 0.0006453990936279297 nb_pixel_total : 15149 time to create 1 rle with old method : 0.020367145538330078 length of segment : 156 time for calcul the mask position with numpy : 0.0036132335662841797 nb_pixel_total : 111355 time to create 1 rle with old method : 0.13305902481079102 length of segment : 497 time for calcul the mask position with numpy : 0.0013172626495361328 nb_pixel_total : 4517 time to create 1 rle with old method : 0.0058879852294921875 length of segment : 100 time for calcul the mask position with numpy : 0.0028870105743408203 nb_pixel_total : 64649 time to create 1 rle with old method : 0.07535934448242188 length of segment : 341 time for calcul the mask position with numpy : 0.0008544921875 nb_pixel_total : 21907 time to create 1 rle with old method : 0.028230667114257812 length of segment : 164 time for calcul the mask position with numpy : 0.00010800361633300781 nb_pixel_total : 1845 time to create 1 rle with old method : 0.002323627471923828 length of segment : 60 time for calcul the mask position with numpy : 0.004023551940917969 nb_pixel_total : 203012 time to create 1 rle with new method : 0.008539676666259766 length of segment : 437 time for calcul the mask position with numpy : 0.011264801025390625 nb_pixel_total : 245215 time to create 1 rle with new method : 0.030334949493408203 length of segment : 974 time for calcul the mask position with numpy : 0.000270843505859375 nb_pixel_total : 2430 time to create 1 rle with old method : 0.0034499168395996094 length of segment : 57 time for calcul the mask position with numpy : 0.0002961158752441406 nb_pixel_total : 3476 time to create 1 rle with old method : 0.004359722137451172 length of segment : 108 time for calcul the mask position with numpy : 0.0029973983764648438 nb_pixel_total : 102659 time to create 1 rle with old method : 0.17154288291931152 length of segment : 427 time for calcul the mask position with numpy : 0.0009443759918212891 nb_pixel_total : 11213 time to create 1 rle with old method : 0.015974044799804688 length of segment : 172 time for calcul the mask position with numpy : 0.0008738040924072266 nb_pixel_total : 15265 time to create 1 rle with old method : 0.018407821655273438 length of segment : 191 time for calcul the mask position with numpy : 0.0001652240753173828 nb_pixel_total : 1656 time to create 1 rle with old method : 0.0023097991943359375 length of segment : 31 time for calcul the mask position with numpy : 0.00030350685119628906 nb_pixel_total : 9321 time to create 1 rle with old method : 0.011461257934570312 length of segment : 77 time for calcul the mask position with numpy : 0.0016944408416748047 nb_pixel_total : 64876 time to create 1 rle with old method : 0.08329153060913086 length of segment : 536 time for calcul the mask position with numpy : 0.00028705596923828125 nb_pixel_total : 3834 time to create 1 rle with old method : 0.004823207855224609 length of segment : 105 time for calcul the mask position with numpy : 0.00532841682434082 nb_pixel_total : 150186 time to create 1 rle with new method : 0.012796401977539062 length of segment : 805 time for calcul the mask position with numpy : 0.003514528274536133 nb_pixel_total : 108222 time to create 1 rle with old method : 0.13646864891052246 length of segment : 331 time for calcul the mask position with numpy : 0.005326509475708008 nb_pixel_total : 171474 time to create 1 rle with new method : 0.013668537139892578 length of segment : 1160 time for calcul the mask position with numpy : 0.00020503997802734375 nb_pixel_total : 1699 time to create 1 rle with old method : 0.0021066665649414062 length of segment : 86 time for calcul the mask position with numpy : 0.001461029052734375 nb_pixel_total : 19098 time to create 1 rle with old method : 0.028150320053100586 length of segment : 168 time for calcul the mask position with numpy : 0.0012540817260742188 nb_pixel_total : 18351 time to create 1 rle with old method : 0.031755924224853516 length of segment : 176 time for calcul the mask position with numpy : 0.0007417201995849609 nb_pixel_total : 5708 time to create 1 rle with old method : 0.010488748550415039 length of segment : 145 time for calcul the mask position with numpy : 0.012499809265136719 nb_pixel_total : 192472 time to create 1 rle with new method : 0.014720439910888672 length of segment : 634 time for calcul the mask position with numpy : 0.001149892807006836 nb_pixel_total : 32374 time to create 1 rle with old method : 0.03790616989135742 length of segment : 198 time for calcul the mask position with numpy : 0.006645917892456055 nb_pixel_total : 181805 time to create 1 rle with new method : 0.0131683349609375 length of segment : 911 time for calcul the mask position with numpy : 0.0030426979064941406 nb_pixel_total : 77507 time to create 1 rle with old method : 0.0891413688659668 length of segment : 317 time for calcul the mask position with numpy : 0.003106355667114258 nb_pixel_total : 65575 time to create 1 rle with old method : 0.08326292037963867 length of segment : 370 time for calcul the mask position with numpy : 0.0071942806243896484 nb_pixel_total : 212929 time to create 1 rle with new method : 0.015206098556518555 length of segment : 654 time for calcul the mask position with numpy : 0.0015811920166015625 nb_pixel_total : 43125 time to create 1 rle with old method : 0.050518035888671875 length of segment : 250 time for calcul the mask position with numpy : 0.0003688335418701172 nb_pixel_total : 4524 time to create 1 rle with old method : 0.0055065155029296875 length of segment : 124 time for calcul the mask position with numpy : 0.0002181529998779297 nb_pixel_total : 1543 time to create 1 rle with old method : 0.002495288848876953 length of segment : 28 time for calcul the mask position with numpy : 0.00014662742614746094 nb_pixel_total : 2034 time to create 1 rle with old method : 0.0026307106018066406 length of segment : 49 time for calcul the mask position with numpy : 0.0013637542724609375 nb_pixel_total : 77352 time to create 1 rle with old method : 0.08929300308227539 length of segment : 311 time for calcul the mask position with numpy : 0.009855031967163086 nb_pixel_total : 290661 time to create 1 rle with new method : 0.019429683685302734 length of segment : 987 time spent for convertir_results : 17.451412439346313 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 162 chid ids of type : 3663 ++++++++++++++++++++++++++Number RLEs to save : 56567 save missing photos in datou_result : time spend for datou_step_exec : 95.4124596118927 time spend to save output : 12.776996612548828 total time spend for step 1 : 108.18945622444153 step2:crop_condition Tue Mar 11 00:48:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 162 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 21 About to insert : list_path_to_insert length 21 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 43 About to insert : list_path_to_insert length 43 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1342684754, 1342684741, 1342684711, 1342684707, 1342684703, 1342684467, 1342684427, 1342684340, 1342684297, 1342515634] Looping around the photos to save general results len do output : 97 /-3703818387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3703818508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3702943339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684754', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684741', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684711', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684707', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684703', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684467', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684427', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684297', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342515634', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 301 time used for this insertion : 0.04101443290710449 save_final save missing photos in datou_result : time spend for datou_step_exec : 60.17281770706177 time spend to save output : 0.043462276458740234 total time spend for step 2 : 60.21627998352051 step3:thcl Tue Mar 11 00:49:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.0747981071472168 time to convert the images to numpy array : 0.6658327579498291 time to import caffe and check if the image exist : 0.07207512855529785 time to convert the images to numpy array : 0.6700811386108398 time to import caffe and check if the image exist : 0.06810212135314941 time to convert the images to numpy array : 0.7103326320648193 time to import caffe and check if the image exist : 0.07294774055480957 time to convert the images to numpy array : 0.7076761722564697 time to import caffe and check if the image exist : 0.07022619247436523 time to convert the images to numpy array : 0.7190520763397217 time to import caffe and check if the image exist : 0.07017254829406738 time to convert the images to numpy array : 0.7414658069610596 time to import caffe and check if the image exist : 0.047522783279418945 time to convert the images to numpy array : 0.7785549163818359 time to import caffe and check if the image exist : 0.0746762752532959 time to convert the images to numpy array : 0.7568156719207764 time to import caffe and check if the image exist : 0.07808566093444824 time to convert the images to numpy array : 0.818422794342041 time to import caffe and check if the image exist : 0.06915974617004395 time to convert the images to numpy array : 0.8311924934387207 total time to convert the images to numpy array : 1.9773712158203125 list photo_ids error: [] list photo_ids correct : [-3703818475, -3703818485, -3703818489, -3703818477, -3703818479, -3703818503, -3703818499, -3703818507, -3703818513, -3703818498, -3703818501, -3703818512, -3703818504, -3703818514, -3703818511, -3703818500, -3703818520, -3702943334, -3703818528, -3702943340, -3703818432, -3703818434, -3703818473, -3703818476, -3703818508, -3702943329, -3702943339, -3702943330, -3703818526, -3702943336, -3703818422, -3703818426, -3703818455, -3703818471, -3702943335, -3703818410, -3703818431, -3703818456, -3703818484, -3703818510, -3702943341, -3702943337, -3702943331, -3703818402, -3703818397, -3703818444, -3703818509, -3703818417, -3703818441, -3703818449, -3703818453, -3703818480, -3703818494, -3703818495, -3703818496, -3703818516, -3703818517, -3703818387, -3703818394, -3703818392, -3703818391, -3703818388, -3703818389, -3703818398, -3703818412, -3703818408, -3703818414, -3703818396, -3703818404, -3703818399, -3703818406, -3703818420, -3703818419, -3703818421, -3703818438, -3703818440, -3703818443, -3703818439, -3703818437, -3703818435, -3703818442, -3703818459, -3703818454, -3703818460, -3703818450, -3703818464, -3703818470, -3703818522, -3703818395, -3703818401, -3703818425, -3703818430, -3703818429, -3703818423, -3703818446, -3703818461, -3703818452] number of photos to traite : 97 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3734 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3515 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.6387543678283691 time used to do the prediction : 0.2945981025695801 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3513 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3513 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.6286864280700684 time used to do the prediction : 0.18441367149353027 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.1457672119140625e-05 save missing photos in datou_result : time spend for datou_step_exec : 13.759252548217773 time spend to save output : 0.0009837150573730469 total time spend for step 3 : 13.760236263275146 step4:merge_mask_thcl_custom Tue Mar 11 00:49:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 162 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1342684754, 1342684741, 1342684711, 1342684707, 1342684703, 1342684467, 1342684427, 1342684340, 1342684297, 1342515634] Looping around the photos to save general results len do output : 10 /1342684754Didn't retrieve data .Didn't retrieve data . /1342684741Didn't retrieve data .Didn't retrieve data . /1342684711Didn't retrieve data .Didn't retrieve data . /1342684707Didn't retrieve data .Didn't retrieve data . /1342684703Didn't retrieve data .Didn't retrieve data . /1342684467Didn't retrieve data .Didn't retrieve data . /1342684427Didn't retrieve data .Didn't retrieve data . /1342684340Didn't retrieve data .Didn't retrieve data . /1342684297Didn't retrieve data .Didn't retrieve data . /1342515634Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684754', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684741', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684711', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684707', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684703', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684467', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684427', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684297', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342515634', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.021182537078857422 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06278347969055176 time spend to save output : 0.021812915802001953 total time spend for step 4 : 0.08459639549255371 step5:rle_unique_nms_with_priority Tue Mar 11 00:49:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 19 nb_hashtags : 7 time to prepare the origin masks : 7.534677982330322 time for calcul the mask position with numpy : 0.32070469856262207 nb_pixel_total : 5094107 time to create 1 rle with new method : 0.6705539226531982 time for calcul the mask position with numpy : 0.02589583396911621 nb_pixel_total : 6339 time to create 1 rle with old method : 0.009302377700805664 time for calcul the mask position with numpy : 0.027328968048095703 nb_pixel_total : 25664 time to create 1 rle with old method : 0.037042856216430664 time for calcul the mask position with numpy : 0.025872468948364258 nb_pixel_total : 349 time to create 1 rle with old method : 0.0007703304290771484 time for calcul the mask position with numpy : 0.04156136512756348 nb_pixel_total : 152606 time to create 1 rle with new method : 0.4883878231048584 time for calcul the mask position with numpy : 0.04753255844116211 nb_pixel_total : 385770 time to create 1 rle with new method : 1.073150873184204 time for calcul the mask position with numpy : 0.05177807807922363 nb_pixel_total : 26496 time to create 1 rle with old method : 0.04118013381958008 time for calcul the mask position with numpy : 0.05038285255432129 nb_pixel_total : 112600 time to create 1 rle with old method : 0.1820845603942871 time for calcul the mask position with numpy : 0.041886091232299805 nb_pixel_total : 176538 time to create 1 rle with new method : 0.4347379207611084 time for calcul the mask position with numpy : 0.035532474517822266 nb_pixel_total : 18676 time to create 1 rle with old method : 0.02156352996826172 time for calcul the mask position with numpy : 0.03543591499328613 nb_pixel_total : 5154 time to create 1 rle with old method : 0.0059816837310791016 time for calcul the mask position with numpy : 0.040158748626708984 nb_pixel_total : 122161 time to create 1 rle with old method : 0.1408390998840332 time for calcul the mask position with numpy : 0.04066610336303711 nb_pixel_total : 49288 time to create 1 rle with old method : 0.05667853355407715 time for calcul the mask position with numpy : 0.03702116012573242 nb_pixel_total : 138747 time to create 1 rle with old method : 0.21058082580566406 time for calcul the mask position with numpy : 0.03614497184753418 nb_pixel_total : 151256 time to create 1 rle with new method : 0.37157344818115234 time for calcul the mask position with numpy : 0.03820037841796875 nb_pixel_total : 219398 time to create 1 rle with new method : 0.3759620189666748 time for calcul the mask position with numpy : 0.028446197509765625 nb_pixel_total : 59856 time to create 1 rle with old method : 0.07732248306274414 time for calcul the mask position with numpy : 0.02639484405517578 nb_pixel_total : 16784 time to create 1 rle with old method : 0.02129507064819336 time for calcul the mask position with numpy : 0.03163719177246094 nb_pixel_total : 69679 time to create 1 rle with old method : 0.0934147834777832 time for calcul the mask position with numpy : 0.040750741958618164 nb_pixel_total : 218772 time to create 1 rle with new method : 0.767188549041748 create new chi : 6.301396131515503 time to delete rle : 0.0195162296295166 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 16294 TO DO : save crop sub photo not yet done ! save time : 2.432253837585449 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 4.111765623092651 time for calcul the mask position with numpy : 0.1600184440612793 nb_pixel_total : 3819534 time to create 1 rle with new method : 0.4307281970977783 time for calcul the mask position with numpy : 0.051374197006225586 nb_pixel_total : 81463 time to create 1 rle with old method : 0.11950373649597168 time for calcul the mask position with numpy : 0.024615049362182617 nb_pixel_total : 100801 time to create 1 rle with old method : 0.11464309692382812 time for calcul the mask position with numpy : 0.022897720336914062 nb_pixel_total : 154441 time to create 1 rle with new method : 0.4613149166107178 time for calcul the mask position with numpy : 0.03030538558959961 nb_pixel_total : 14981 time to create 1 rle with old method : 0.01775956153869629 time for calcul the mask position with numpy : 0.3149747848510742 nb_pixel_total : 2879020 time to create 1 rle with new method : 0.3290257453918457 create new chi : 2.166882038116455 time to delete rle : 0.0011951923370361328 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 9905 TO DO : save crop sub photo not yet done ! save time : 3.8586623668670654 nb_obj : 17 nb_hashtags : 7 time to prepare the origin masks : 9.874172449111938 time for calcul the mask position with numpy : 0.8671531677246094 nb_pixel_total : 5891440 time to create 1 rle with new method : 0.40514683723449707 time for calcul the mask position with numpy : 0.04044079780578613 nb_pixel_total : 30697 time to create 1 rle with old method : 0.03627896308898926 time for calcul the mask position with numpy : 0.035668373107910156 nb_pixel_total : 51766 time to create 1 rle with old method : 0.05883193016052246 time for calcul the mask position with numpy : 0.035547733306884766 nb_pixel_total : 73724 time to create 1 rle with old method : 0.08267998695373535 time for calcul the mask position with numpy : 0.03287458419799805 nb_pixel_total : 17246 time to create 1 rle with old method : 0.02088475227355957 time for calcul the mask position with numpy : 0.0326998233795166 nb_pixel_total : 194207 time to create 1 rle with new method : 2.300969123840332 time for calcul the mask position with numpy : 0.04785752296447754 nb_pixel_total : 114130 time to create 1 rle with old method : 0.1467583179473877 time for calcul the mask position with numpy : 0.027581214904785156 nb_pixel_total : 7542 time to create 1 rle with old method : 0.00905752182006836 time for calcul the mask position with numpy : 0.026221752166748047 nb_pixel_total : 9040 time to create 1 rle with old method : 0.01838827133178711 time for calcul the mask position with numpy : 0.04960060119628906 nb_pixel_total : 6590 time to create 1 rle with old method : 0.012036561965942383 time for calcul the mask position with numpy : 0.0489649772644043 nb_pixel_total : 166071 time to create 1 rle with new method : 0.9772288799285889 time for calcul the mask position with numpy : 0.03837990760803223 nb_pixel_total : 26727 time to create 1 rle with old method : 0.04843640327453613 time for calcul the mask position with numpy : 0.03822779655456543 nb_pixel_total : 30277 time to create 1 rle with old method : 0.04555344581604004 time for calcul the mask position with numpy : 0.036108970642089844 nb_pixel_total : 7268 time to create 1 rle with old method : 0.010873556137084961 time for calcul the mask position with numpy : 0.039548635482788086 nb_pixel_total : 137451 time to create 1 rle with old method : 0.18739891052246094 time for calcul the mask position with numpy : 0.045282840728759766 nb_pixel_total : 3902 time to create 1 rle with old method : 0.005255699157714844 time for calcul the mask position with numpy : 0.0419316291809082 nb_pixel_total : 16560 time to create 1 rle with old method : 0.026716232299804688 time for calcul the mask position with numpy : 0.045676469802856445 nb_pixel_total : 265602 time to create 1 rle with new method : 1.2925641536712646 create new chi : 7.361508131027222 time to delete rle : 0.001577138900756836 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 13131 TO DO : save crop sub photo not yet done ! save time : 5.538928270339966 nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 16.987606525421143 time for calcul the mask position with numpy : 1.3303256034851074 nb_pixel_total : 5490927 time to create 1 rle with new method : 1.7291197776794434 time for calcul the mask position with numpy : 0.04601407051086426 nb_pixel_total : 120 time to create 1 rle with old method : 0.0009164810180664062 time for calcul the mask position with numpy : 0.04621267318725586 nb_pixel_total : 18116 time to create 1 rle with old method : 0.02225971221923828 time for calcul the mask position with numpy : 0.037833213806152344 nb_pixel_total : 16824 time to create 1 rle with old method : 0.023175954818725586 time for calcul the mask position with numpy : 0.04736971855163574 nb_pixel_total : 8848 time to create 1 rle with old method : 0.010195732116699219 time for calcul the mask position with numpy : 0.0379643440246582 nb_pixel_total : 894 time to create 1 rle with old method : 0.0011148452758789062 time for calcul the mask position with numpy : 0.04034233093261719 nb_pixel_total : 75524 time to create 1 rle with old method : 0.09240412712097168 time for calcul the mask position with numpy : 0.03881716728210449 nb_pixel_total : 2151 time to create 1 rle with old method : 0.0026586055755615234 time for calcul the mask position with numpy : 0.04257655143737793 nb_pixel_total : 252461 time to create 1 rle with new method : 0.7689688205718994 time for calcul the mask position with numpy : 0.03939056396484375 nb_pixel_total : 1129 time to create 1 rle with old method : 0.0013680458068847656 time for calcul the mask position with numpy : 0.03803563117980957 nb_pixel_total : 78651 time to create 1 rle with old method : 0.11884474754333496 time for calcul the mask position with numpy : 0.04178905487060547 nb_pixel_total : 29120 time to create 1 rle with old method : 0.035523414611816406 time for calcul the mask position with numpy : 0.044405460357666016 nb_pixel_total : 226112 time to create 1 rle with new method : 0.8687551021575928 time for calcul the mask position with numpy : 0.04160332679748535 nb_pixel_total : 12053 time to create 1 rle with old method : 0.013660669326782227 time for calcul the mask position with numpy : 0.04010319709777832 nb_pixel_total : 271334 time to create 1 rle with new method : 1.7596166133880615 time for calcul the mask position with numpy : 0.04242229461669922 nb_pixel_total : 202569 time to create 1 rle with new method : 1.981004238128662 time for calcul the mask position with numpy : 0.04877901077270508 nb_pixel_total : 269174 time to create 1 rle with new method : 2.075493574142456 time for calcul the mask position with numpy : 0.03859353065490723 nb_pixel_total : 94233 time to create 1 rle with old method : 0.10924673080444336 create new chi : 11.840642213821411 time to delete rle : 0.002290964126586914 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 15560 TO DO : save crop sub photo not yet done ! save time : 1.277456283569336 nb_obj : 13 nb_hashtags : 6 time to prepare the origin masks : 13.929979085922241 time for calcul the mask position with numpy : 0.8609869480133057 nb_pixel_total : 6377006 time to create 1 rle with new method : 0.7543013095855713 time for calcul the mask position with numpy : 0.042793989181518555 nb_pixel_total : 9501 time to create 1 rle with old method : 0.012565135955810547 time for calcul the mask position with numpy : 0.04227852821350098 nb_pixel_total : 20415 time to create 1 rle with old method : 0.023191452026367188 time for calcul the mask position with numpy : 0.04724740982055664 nb_pixel_total : 1043 time to create 1 rle with old method : 0.0017180442810058594 time for calcul the mask position with numpy : 0.04601168632507324 nb_pixel_total : 2348 time to create 1 rle with old method : 0.0034401416778564453 time for calcul the mask position with numpy : 0.05221843719482422 nb_pixel_total : 296984 time to create 1 rle with new method : 0.6992785930633545 time for calcul the mask position with numpy : 0.04364132881164551 nb_pixel_total : 22756 time to create 1 rle with old method : 0.02704453468322754 time for calcul the mask position with numpy : 0.04275012016296387 nb_pixel_total : 5181 time to create 1 rle with old method : 0.005966663360595703 time for calcul the mask position with numpy : 0.0402369499206543 nb_pixel_total : 708 time to create 1 rle with old method : 0.0014393329620361328 time for calcul the mask position with numpy : 0.04061746597290039 nb_pixel_total : 72433 time to create 1 rle with old method : 0.08844661712646484 time for calcul the mask position with numpy : 0.04096698760986328 nb_pixel_total : 58586 time to create 1 rle with old method : 0.08398246765136719 time for calcul the mask position with numpy : 0.04126715660095215 nb_pixel_total : 4994 time to create 1 rle with old method : 0.005817890167236328 time for calcul the mask position with numpy : 0.02591252326965332 nb_pixel_total : 125077 time to create 1 rle with old method : 0.15134286880493164 time for calcul the mask position with numpy : 0.040843963623046875 nb_pixel_total : 53208 time to create 1 rle with old method : 0.06324601173400879 create new chi : 3.4065937995910645 time to delete rle : 0.0010981559753417969 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 9805 TO DO : save crop sub photo not yet done ! save time : 2.6558003425598145 nb_obj : 15 nb_hashtags : 6 time to prepare the origin masks : 10.976314306259155 time for calcul the mask position with numpy : 0.30669355392456055 nb_pixel_total : 5244145 time to create 1 rle with new method : 0.4769144058227539 time for calcul the mask position with numpy : 0.03329753875732422 nb_pixel_total : 5373 time to create 1 rle with old method : 0.0061571598052978516 time for calcul the mask position with numpy : 0.03607797622680664 nb_pixel_total : 361519 time to create 1 rle with new method : 1.05985426902771 time for calcul the mask position with numpy : 0.026566267013549805 nb_pixel_total : 11437 time to create 1 rle with old method : 0.013277292251586914 time for calcul the mask position with numpy : 0.02734994888305664 nb_pixel_total : 167693 time to create 1 rle with new method : 0.6562354564666748 time for calcul the mask position with numpy : 0.05783796310424805 nb_pixel_total : 5058 time to create 1 rle with old method : 0.006650447845458984 time for calcul the mask position with numpy : 0.0509796142578125 nb_pixel_total : 18375 time to create 1 rle with old method : 0.02217555046081543 time for calcul the mask position with numpy : 0.053578853607177734 nb_pixel_total : 604496 time to create 1 rle with new method : 0.616426944732666 time for calcul the mask position with numpy : 0.058617591857910156 nb_pixel_total : 2710 time to create 1 rle with old method : 0.0044116973876953125 time for calcul the mask position with numpy : 0.056182861328125 nb_pixel_total : 3300 time to create 1 rle with old method : 0.0040760040283203125 time for calcul the mask position with numpy : 0.07573270797729492 nb_pixel_total : 483852 time to create 1 rle with new method : 0.5990734100341797 time for calcul the mask position with numpy : 0.06387948989868164 nb_pixel_total : 21307 time to create 1 rle with old method : 0.02800130844116211 time for calcul the mask position with numpy : 0.04848837852478027 nb_pixel_total : 14799 time to create 1 rle with old method : 0.01729559898376465 time for calcul the mask position with numpy : 0.046889305114746094 nb_pixel_total : 1660 time to create 1 rle with old method : 0.0020241737365722656 time for calcul the mask position with numpy : 0.0429227352142334 nb_pixel_total : 81776 time to create 1 rle with old method : 0.09814119338989258 time for calcul the mask position with numpy : 0.041222572326660156 nb_pixel_total : 22740 time to create 1 rle with old method : 0.026041030883789062 create new chi : 4.915936231613159 time to delete rle : 0.0018608570098876953 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 12552 TO DO : save crop sub photo not yet done ! save time : 0.9854278564453125 nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 12.67865777015686 time for calcul the mask position with numpy : 0.6223828792572021 nb_pixel_total : 6469826 time to create 1 rle with new method : 0.7430846691131592 time for calcul the mask position with numpy : 0.06082558631896973 nb_pixel_total : 55553 time to create 1 rle with old method : 0.06998491287231445 time for calcul the mask position with numpy : 0.04753398895263672 nb_pixel_total : 14466 time to create 1 rle with old method : 0.01543426513671875 time for calcul the mask position with numpy : 0.04317927360534668 nb_pixel_total : 8477 time to create 1 rle with old method : 0.009538888931274414 time for calcul the mask position with numpy : 0.04457569122314453 nb_pixel_total : 1603 time to create 1 rle with old method : 0.001766204833984375 time for calcul the mask position with numpy : 0.04537510871887207 nb_pixel_total : 3406 time to create 1 rle with old method : 0.0038237571716308594 time for calcul the mask position with numpy : 0.050073862075805664 nb_pixel_total : 5539 time to create 1 rle with old method : 0.0058934688568115234 time for calcul the mask position with numpy : 0.04759645462036133 nb_pixel_total : 26265 time to create 1 rle with old method : 0.028484344482421875 time for calcul the mask position with numpy : 0.03018021583557129 nb_pixel_total : 5240 time to create 1 rle with old method : 0.005745649337768555 time for calcul the mask position with numpy : 0.03220081329345703 nb_pixel_total : 81219 time to create 1 rle with old method : 0.11253166198730469 time for calcul the mask position with numpy : 0.03313040733337402 nb_pixel_total : 3541 time to create 1 rle with old method : 0.004361867904663086 time for calcul the mask position with numpy : 0.03691434860229492 nb_pixel_total : 12247 time to create 1 rle with old method : 0.01336669921875 time for calcul the mask position with numpy : 0.03315401077270508 nb_pixel_total : 4971 time to create 1 rle with old method : 0.00680232048034668 time for calcul the mask position with numpy : 0.028652667999267578 nb_pixel_total : 3103 time to create 1 rle with old method : 0.003496408462524414 time for calcul the mask position with numpy : 0.04161882400512695 nb_pixel_total : 23299 time to create 1 rle with old method : 0.026287317276000977 time for calcul the mask position with numpy : 0.05879974365234375 nb_pixel_total : 283931 time to create 1 rle with new method : 0.5541276931762695 time for calcul the mask position with numpy : 0.04797482490539551 nb_pixel_total : 1057 time to create 1 rle with old method : 0.0012929439544677734 time for calcul the mask position with numpy : 0.055027008056640625 nb_pixel_total : 46497 time to create 1 rle with old method : 0.05418729782104492 create new chi : 3.0930063724517822 time to delete rle : 0.0010647773742675781 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 8950 TO DO : save crop sub photo not yet done ! save time : 1.7945556640625 nb_obj : 22 nb_hashtags : 6 time to prepare the origin masks : 13.018651247024536 time for calcul the mask position with numpy : 0.4783782958984375 nb_pixel_total : 5865615 time to create 1 rle with new method : 0.6314771175384521 time for calcul the mask position with numpy : 0.04688453674316406 nb_pixel_total : 1836 time to create 1 rle with old method : 0.0026564598083496094 time for calcul the mask position with numpy : 0.04615974426269531 nb_pixel_total : 32641 time to create 1 rle with old method : 0.04433894157409668 time for calcul the mask position with numpy : 0.026676416397094727 nb_pixel_total : 4659 time to create 1 rle with old method : 0.006417036056518555 time for calcul the mask position with numpy : 0.025567054748535156 nb_pixel_total : 16314 time to create 1 rle with old method : 0.019910335540771484 time for calcul the mask position with numpy : 0.02652144432067871 nb_pixel_total : 17554 time to create 1 rle with old method : 0.022201061248779297 time for calcul the mask position with numpy : 0.026784896850585938 nb_pixel_total : 21857 time to create 1 rle with old method : 0.027998685836791992 time for calcul the mask position with numpy : 0.027413368225097656 nb_pixel_total : 24885 time to create 1 rle with old method : 0.048287153244018555 time for calcul the mask position with numpy : 0.0339968204498291 nb_pixel_total : 94724 time to create 1 rle with old method : 0.15459299087524414 time for calcul the mask position with numpy : 0.04128742218017578 nb_pixel_total : 3346 time to create 1 rle with old method : 0.005181074142456055 time for calcul the mask position with numpy : 0.0366363525390625 nb_pixel_total : 2066 time to create 1 rle with old method : 0.003262042999267578 time for calcul the mask position with numpy : 0.03647351264953613 nb_pixel_total : 15119 time to create 1 rle with old method : 0.02343583106994629 time for calcul the mask position with numpy : 0.03204822540283203 nb_pixel_total : 5013 time to create 1 rle with old method : 0.00695037841796875 time for calcul the mask position with numpy : 0.031397104263305664 nb_pixel_total : 16759 time to create 1 rle with old method : 0.02044963836669922 time for calcul the mask position with numpy : 0.028151512145996094 nb_pixel_total : 170 time to create 1 rle with old method : 0.0005145072937011719 time for calcul the mask position with numpy : 0.031087875366210938 nb_pixel_total : 107308 time to create 1 rle with old method : 0.14065861701965332 time for calcul the mask position with numpy : 0.03271055221557617 nb_pixel_total : 64276 time to create 1 rle with old method : 0.07633519172668457 time for calcul the mask position with numpy : 0.03134274482727051 nb_pixel_total : 171689 time to create 1 rle with new method : 1.0618855953216553 time for calcul the mask position with numpy : 0.027620553970336914 nb_pixel_total : 67166 time to create 1 rle with old method : 0.09197115898132324 time for calcul the mask position with numpy : 0.038360595703125 nb_pixel_total : 295548 time to create 1 rle with new method : 0.5043935775756836 time for calcul the mask position with numpy : 0.0279386043548584 nb_pixel_total : 4500 time to create 1 rle with old method : 0.005228281021118164 time for calcul the mask position with numpy : 0.02884507179260254 nb_pixel_total : 101998 time to create 1 rle with old method : 0.12887191772460938 time for calcul the mask position with numpy : 0.026438474655151367 nb_pixel_total : 115197 time to create 1 rle with old method : 0.1308002471923828 create new chi : 4.457318305969238 time to delete rle : 0.001664876937866211 batch 1 Loaded 23 chid ids of type : 3726 Number RLEs to save : 13438 TO DO : save crop sub photo not yet done ! save time : 6.737541198730469 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 5.8193371295928955 time for calcul the mask position with numpy : 0.4356191158294678 nb_pixel_total : 6457890 time to create 1 rle with new method : 0.7277231216430664 time for calcul the mask position with numpy : 0.0373685359954834 nb_pixel_total : 9275 time to create 1 rle with old method : 0.010852336883544922 time for calcul the mask position with numpy : 0.04036450386047363 nb_pixel_total : 15189 time to create 1 rle with old method : 0.02500319480895996 time for calcul the mask position with numpy : 0.03841972351074219 nb_pixel_total : 1640 time to create 1 rle with old method : 0.002680540084838867 time for calcul the mask position with numpy : 0.03725624084472656 nb_pixel_total : 11166 time to create 1 rle with old method : 0.012513160705566406 time for calcul the mask position with numpy : 0.03720664978027344 nb_pixel_total : 102298 time to create 1 rle with old method : 0.11848068237304688 time for calcul the mask position with numpy : 0.03536105155944824 nb_pixel_total : 3464 time to create 1 rle with old method : 0.017991065979003906 time for calcul the mask position with numpy : 0.037180185317993164 nb_pixel_total : 202677 time to create 1 rle with new method : 0.5356099605560303 time for calcul the mask position with numpy : 0.032586097717285156 nb_pixel_total : 2414 time to create 1 rle with old method : 0.0027246475219726562 time for calcul the mask position with numpy : 0.0243680477142334 nb_pixel_total : 244227 time to create 1 rle with new method : 0.3168299198150635 create new chi : 2.6160366535186768 time to delete rle : 0.0010221004486083984 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 7156 TO DO : save crop sub photo not yet done ! save time : 3.718947172164917 nb_obj : 19 nb_hashtags : 5 time to prepare the origin masks : 12.238027811050415 time for calcul the mask position with numpy : 0.42861390113830566 nb_pixel_total : 5691815 time to create 1 rle with new method : 1.3540620803833008 time for calcul the mask position with numpy : 0.04268622398376465 nb_pixel_total : 54751 time to create 1 rle with old method : 0.0643014907836914 time for calcul the mask position with numpy : 0.04025411605834961 nb_pixel_total : 77288 time to create 1 rle with old method : 0.09001946449279785 time for calcul the mask position with numpy : 0.0413365364074707 nb_pixel_total : 201140 time to create 1 rle with new method : 0.5856225490570068 time for calcul the mask position with numpy : 0.029634952545166016 nb_pixel_total : 149582 time to create 1 rle with old method : 0.17723798751831055 time for calcul the mask position with numpy : 0.026378393173217773 nb_pixel_total : 28 time to create 1 rle with old method : 0.00010538101196289062 time for calcul the mask position with numpy : 0.030319929122924805 nb_pixel_total : 2053 time to create 1 rle with old method : 0.002355337142944336 time for calcul the mask position with numpy : 0.025113582611083984 nb_pixel_total : 528 time to create 1 rle with old method : 0.0006973743438720703 time for calcul the mask position with numpy : 0.022406339645385742 nb_pixel_total : 65272 time to create 1 rle with old method : 0.07715439796447754 time for calcul the mask position with numpy : 0.023313045501708984 nb_pixel_total : 4513 time to create 1 rle with old method : 0.005785942077636719 time for calcul the mask position with numpy : 0.02319955825805664 nb_pixel_total : 192429 time to create 1 rle with new method : 0.8314673900604248 time for calcul the mask position with numpy : 0.022031068801879883 nb_pixel_total : 5690 time to create 1 rle with old method : 0.006507158279418945 time for calcul the mask position with numpy : 0.021565675735473633 nb_pixel_total : 60411 time to create 1 rle with old method : 0.0671999454498291 time for calcul the mask position with numpy : 0.023525714874267578 nb_pixel_total : 32217 time to create 1 rle with old method : 0.03601527214050293 time for calcul the mask position with numpy : 0.02149820327758789 nb_pixel_total : 18958 time to create 1 rle with old method : 0.021844148635864258 time for calcul the mask position with numpy : 0.022053956985473633 nb_pixel_total : 1698 time to create 1 rle with old method : 0.0020492076873779297 time for calcul the mask position with numpy : 0.026043415069580078 nb_pixel_total : 288281 time to create 1 rle with new method : 0.42988133430480957 time for calcul the mask position with numpy : 0.02183818817138672 nb_pixel_total : 18307 time to create 1 rle with old method : 0.027101516723632812 time for calcul the mask position with numpy : 0.023013830184936523 nb_pixel_total : 3818 time to create 1 rle with old method : 0.006162405014038086 time for calcul the mask position with numpy : 0.02427196502685547 nb_pixel_total : 181461 time to create 1 rle with new method : 0.5435788631439209 create new chi : 5.407849550247192 time to delete rle : 0.0016536712646484375 batch 1 Loaded 23 chid ids of type : 3726 Number RLEs to save : 15387 TO DO : save crop sub photo not yet done ! save time : 7.411898136138916 map_output_result : {1342684754: (0.0, 'Should be the crop_list due to order', 0), 1342684741: (0.0, 'Should be the crop_list due to order', 0), 1342684711: (0.0, 'Should be the crop_list due to order', 0), 1342684707: (0.0, 'Should be the crop_list due to order', 0), 1342684703: (0.0, 'Should be the crop_list due to order', 0), 1342684467: (0.0, 'Should be the crop_list due to order', 0), 1342684427: (0.0, 'Should be the crop_list due to order', 0), 1342684340: (0.0, 'Should be the crop_list due to order', 0), 1342684297: (0.0, 'Should be the crop_list due to order', 0), 1342515634: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1342684754, 1342684741, 1342684711, 1342684707, 1342684703, 1342684467, 1342684427, 1342684340, 1342684297, 1342515634] Looping around the photos to save general results len do output : 10 /1342684754.Didn't retrieve data . /1342684741.Didn't retrieve data . /1342684711.Didn't retrieve data . /1342684707.Didn't retrieve data . /1342684703.Didn't retrieve data . /1342684467.Didn't retrieve data . /1342684427.Didn't retrieve data . /1342684340.Didn't retrieve data . /1342684297.Didn't retrieve data . /1342515634.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684754', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684741', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684711', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684707', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684703', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684467', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684427', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684297', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342515634', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01588606834411621 save_final save missing photos in datou_result : time spend for datou_step_exec : 196.36132311820984 time spend to save output : 0.06636953353881836 total time spend for step 5 : 196.42769265174866 step6:crop_condition Tue Mar 11 00:52:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 166 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741650774_1798213 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.4742116928100586 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! About to upload 14 photos upload in portfolio : 4869462 init cache_photo without model_param we have 14 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741650785_1798213 we have uploaded 14 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.185149192810059 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741650792_1798213 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.775087594985962 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741650796_1798213 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.340245485305786 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741650804_1798213 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3916854858398438 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741650809_1798213 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.032550573348999 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1342684754, 1342684741, 1342684711, 1342684707, 1342684703, 1342684467, 1342684427, 1342684340, 1342684297, 1342515634] Looping around the photos to save general results len do output : 38 /1342735243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735245Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735420Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1342735448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684754', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684741', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684711', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684707', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684703', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684467', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684427', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684297', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342515634', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 124 time used for this insertion : 0.1631007194519043 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.841238260269165 time spend to save output : 0.168961763381958 total time spend for step 6 : 49.01020002365112 step7:ventilate_hashtags_in_portfolio Tue Mar 11 00:53:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 21247982 get user id for portfolio 21247982 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247982 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','Teint_Dans_La_Masse','Carton_gris','cartonnette','autre_refus','Carton_brun','environnement','flou','papier','kraft','mal_croppe','plastique')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247982 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','Teint_Dans_La_Masse','Carton_gris','cartonnette','autre_refus','Carton_brun','environnement','flou','papier','kraft','mal_croppe','plastique')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247982 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','Teint_Dans_La_Masse','Carton_gris','cartonnette','autre_refus','Carton_brun','environnement','flou','papier','kraft','mal_croppe','plastique')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/21248250,21248251,21248252,21248253,21248254,21248255,21248256,21248257,21248258,21248259,21248260,21248261?tags=metal,Teint_Dans_La_Masse,Carton_gris,cartonnette,autre_refus,Carton_brun,environnement,flou,papier,kraft,mal_croppe,plastique Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1342684754, 1342684741, 1342684711, 1342684707, 1342684703, 1342684467, 1342684427, 1342684340, 1342684297, 1342515634] Looping around the photos to save general results len do output : 1 /21247982. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684754', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684741', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684711', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684707', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684703', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684467', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684427', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684297', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342515634', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.01724696159362793 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.5005288124084473 time spend to save output : 0.017533540725708008 total time spend for step 7 : 3.5180623531341553 step8:final Tue Mar 11 00:53:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1342684754: ('0.07617732921344036',), 1342684741: ('0.07617732921344036',), 1342684711: ('0.07617732921344036',), 1342684707: ('0.07617732921344036',), 1342684703: ('0.07617732921344036',), 1342684467: ('0.07617732921344036',), 1342684427: ('0.07617732921344036',), 1342684340: ('0.07617732921344036',), 1342684297: ('0.07617732921344036',), 1342515634: ('0.07617732921344036',)} new output for save of step final : {1342684754: ('0.07617732921344036',), 1342684741: ('0.07617732921344036',), 1342684711: ('0.07617732921344036',), 1342684707: ('0.07617732921344036',), 1342684703: ('0.07617732921344036',), 1342684467: ('0.07617732921344036',), 1342684427: ('0.07617732921344036',), 1342684340: ('0.07617732921344036',), 1342684297: ('0.07617732921344036',), 1342515634: ('0.07617732921344036',)} [1342684754, 1342684741, 1342684711, 1342684707, 1342684703, 1342684467, 1342684427, 1342684340, 1342684297, 1342515634] Looping around the photos to save general results len do output : 10 /1342684754.Didn't retrieve data . /1342684741.Didn't retrieve data . /1342684711.Didn't retrieve data . /1342684707.Didn't retrieve data . /1342684703.Didn't retrieve data . /1342684467.Didn't retrieve data . /1342684427.Didn't retrieve data . /1342684340.Didn't retrieve data . /1342684297.Didn't retrieve data . /1342515634.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684754', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684741', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684711', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684707', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684703', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684467', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684427', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684297', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342515634', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.012604475021362305 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.21849846839904785 time spend to save output : 0.013110637664794922 total time spend for step 8 : 0.23160910606384277 step9:velours_tree Tue Mar 11 00:53:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.1891026496887207 time spend to save output : 4.076957702636719e-05 total time spend for step 9 : 0.18914341926574707 step10:send_mail_cod Tue Mar 11 00:53:40 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P21247982_11-03-2025_00_53_40.pdf 21248250 imagette212482501741650820 21248253 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482531741650820 21248254 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482541741650820 21248257 imagette212482571741650822 21248259 change filename to text .imagette212482591741650822 21248260 imagette212482601741650822 21248261 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482611741650822 21248251 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482511741650823 21248252 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482521741650824 21248255 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212482551741650825 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21247982 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21248250,21248251,21248252,21248253,21248254,21248255,21248256,21248257,21248258,21248259,21248260,21248261?tags=metal,Teint_Dans_La_Masse,Carton_gris,cartonnette,autre_refus,Carton_brun,environnement,flou,papier,kraft,mal_croppe,plastique your option no_mail is active, we will not send the real mail to your client args[1342684754] : ('0.07617732921344036',) no score found for photo 1342684754 We are sending mail with results at report@fotonower.com args[1342684741] : ('0.07617732921344036',) no score found for photo 1342684741 We are sending mail with results at report@fotonower.com args[1342684711] : ('0.07617732921344036',) no score found for photo 1342684711 We are sending mail with results at report@fotonower.com args[1342684707] : ('0.07617732921344036',) no score found for photo 1342684707 We are sending mail with results at report@fotonower.com args[1342684703] : ('0.07617732921344036',) no score found for photo 1342684703 We are sending mail with results at report@fotonower.com args[1342684467] : ('0.07617732921344036',) no score found for photo 1342684467 We are sending mail with results at report@fotonower.com args[1342684427] : ('0.07617732921344036',) no score found for photo 1342684427 We are sending mail with results at report@fotonower.com args[1342684340] : ('0.07617732921344036',) no score found for photo 1342684340 We are sending mail with results at report@fotonower.com args[1342684297] : ('0.07617732921344036',) no score found for photo 1342684297 We are sending mail with results at report@fotonower.com args[1342515634] : ('0.07617732921344036',) no score found for photo 1342515634 We are sending mail with results at report@fotonower.com refus_total : 0.07617732921344036 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21247982 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1342515634,1342684754,1342684711,1342684707,1342684703,1342684467,1342684427,1342684340,1342684297,1342515140,1342515556,1342515351,1342515347,1342515343,1342515147,1342515144,1342515340,1342515358,1342515354,1342684741) Found this number of photos: 20 begin to download photo : 1342515634 begin to download photo : 1342684467 begin to download photo : 1342515556 begin to download photo : 1342515144 download finish for photo 1342684467 begin to download photo : 1342684427 download finish for photo 1342515144 begin to download photo : 1342515340 download finish for photo 1342515634 begin to download photo : 1342684754 download finish for photo 1342515556 begin to download photo : 1342515351 download finish for photo 1342684754 begin to download photo : 1342684711 download finish for photo 1342515340 begin to download photo : 1342515358 download finish for photo 1342684427 begin to download photo : 1342684340 download finish for photo 1342515351 begin to download photo : 1342515347 download finish for photo 1342515358 begin to download photo : 1342515354 download finish for photo 1342684340 begin to download photo : 1342684297 download finish for photo 1342684711 begin to download photo : 1342684707 download finish for photo 1342684297 begin to download photo : 1342515140 download finish for photo 1342684707 begin to download photo : 1342684703 download finish for photo 1342515347 begin to download photo : 1342515343 download finish for photo 1342515354 begin to download photo : 1342684741 download finish for photo 1342684703 download finish for photo 1342684741 download finish for photo 1342515140 download finish for photo 1342515343 begin to download photo : 1342515147 download finish for photo 1342515147 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247982_11-03-2025_00_53_40.pdf results_Auto_P21247982_11-03-2025_00_53_40.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247982_11-03-2025_00_53_40.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','21247982','results_Auto_P21247982_11-03-2025_00_53_40.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247982_11-03-2025_00_53_40.pdf','pdf','','1.22','0.07617732921344036') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1342684754, 1342684741, 1342684711, 1342684707, 1342684703, 1342684467, 1342684427, 1342684340, 1342684297, 1342515634] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684754', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684741', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684711', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684707', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684703', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684467', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684427', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684297', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342515634', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.01814556121826172 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.155937194824219 time spend to save output : 0.018419504165649414 total time spend for step 10 : 10.174356698989868 step11:split_time_score Tue Mar 11 00:53:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 20),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10032025 21247982 Nombre de photos uploadées : 20 / 23040 (0%) 10032025 21247982 Nombre de photos taguées (types de déchets): 0 / 20 (0%) 10032025 21247982 Nombre de photos taguées (volume) : 0 / 20 (0%) elapsed_time : load_data_split_time_score 2.86102294921875e-06 elapsed_time : order_list_meta_photo_and_scores 7.62939453125e-06 ???????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.001443624496459961 elapsed_time : insert_dashboard_record_day_entry 0.02892160415649414 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244281 order by id desc limit 1 Qualite : 0.20374319704010083 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244282_10-03-2025_23_03_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244282 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244282 AND mptpi.`type`=3594 To do Qualite : 0.18168180090323166 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244283_10-03-2025_22_41_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244283 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244283 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244308_11-03-2025_00_50_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244308 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244309 order by id desc limit 1 Qualite : 0.34839745750897183 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244310_10-03-2025_23_16_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244310 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244310 AND mptpi.`type`=4200 To do Qualite : 0.2737082523489908 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244311_10-03-2025_22_37_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244311 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244311 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247979 order by id desc limit 1 Qualite : 0.06524887220116574 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245755_10-03-2025_23_29_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245755 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245755 AND mptpi.`type`=4207 To do Qualite : 0.072106598848494 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245772_10-03-2025_23_26_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245772 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245772 AND mptpi.`type`=4207 To do Qualite : 0.04447951517284577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245773_10-03-2025_23_08_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245773 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245773 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247980 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247981 order by id desc limit 1 Qualite : 0.07617732921344036 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247982_11-03-2025_00_53_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247982 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247982 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247983 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247984 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247985 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247986 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247987 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247988 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247989 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247990 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247991 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247992 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247993 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222608 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222609 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222611 order by id desc limit 1 Qualite : 0.04897866899276895 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21222613_10-03-2025_17_51_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222613 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222613 AND mptpi.`type`=4209 To do Qualite : 0.05516057268644328 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21222058_10-03-2025_14_57_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222058 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222058 AND mptpi.`type`=3726 To do Qualite : 0.2309571105130506 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21222084_10-03-2025_15_14_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21222084 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21222084 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247994 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247995 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247996 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247997 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247998 order by id desc limit 1 Qualite : 0.06957109304578478 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21247999_11-03-2025_00_53_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21247999 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21247999 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248000 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248001 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248014 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21248019_11-03-2025_00_52_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248019 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248020 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248021 order by id desc limit 1 Qualite : 0.08314390126051041 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245775_10-03-2025_23_04_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245775 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245775 AND mptpi.`type`=3726 To do Qualite : 0.07125952811625422 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245776_10-03-2025_23_20_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245776 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245776 AND mptpi.`type`=4209 To do Qualite : 0.08640782218121312 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245777_10-03-2025_23_45_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245777 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245777 AND mptpi.`type`=4207 To do Qualite : 0.05258407715521776 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245779_10-03-2025_23_16_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245779 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245779 AND mptpi.`type`=4207 To do Qualite : 0.06963896934644463 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245780_10-03-2025_22_56_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245780 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245780 AND mptpi.`type`=4207 To do Qualite : 0.06276088819330108 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21245781_10-03-2025_23_01_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21245781 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21245781 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21236129 order by id desc limit 1 Qualite : 0.2393178491409189 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21236155_10-03-2025_21_12_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21236155 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21236155 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21236158_10-03-2025_19_45_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21236158 order by id desc limit 1 Qualite : 0.2502752435850885 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21236177_10-03-2025_19_58_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21236177 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21236177 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21248022 order by id desc limit 1 Qualite : 0.22092496141975312 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244378_10-03-2025_23_40_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244378 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244378 AND mptpi.`type`=3594 To do Qualite : 0.0657541186557689 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244384_11-03-2025_00_34_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244384 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244384 AND mptpi.`type`=4207 To do Qualite : 0.05471434328258402 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244385_10-03-2025_22_31_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244385 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244385 AND mptpi.`type`=4207 To do Qualite : 0.05735829306752358 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21244386_10-03-2025_22_27_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21244386 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21244386 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10032025': {'nb_upload': 20, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1342684754, 1342684741, 1342684711, 1342684707, 1342684703, 1342684467, 1342684427, 1342684340, 1342684297, 1342515634] Looping around the photos to save general results len do output : 1 /21247982Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684754', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684741', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684711', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684707', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684703', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684467', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684427', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684340', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342684297', None, None, None, None, None, '2656120') ('3459', None, None, None, None, None, None, None, '2656120') ('3459', '21247982', '1342515634', None, None, None, None, None, '2656120') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.04644036293029785 save_final save missing photos in datou_result : time spend for datou_step_exec : 27.721317529678345 time spend to save output : 0.046684980392456055 total time spend for step 11 : 27.7680025100708 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 185.74user 172.38system 7:55.34elapsed 75%CPU (0avgtext+0avgdata 3783284maxresident)k 3251488inputs+167528outputs (28405major+15387821minor)pagefaults 0swaps