python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1014818 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 28>10 please execute split_portfolio.py -i 20176483 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2539822'] with mtr_portfolio_ids : ['20176483'] and first list_photo_ids : [] new path : /proc/1014818/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.9459083080291748 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Mon Feb 3 18:04:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2922 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-03 18:04:33.538177: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-03 18:04:33.563366: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-03 18:04:33.565485: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fed0c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-03 18:04:33.565540: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-03 18:04:33.569445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-03 18:04:33.844488: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x99ab7b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-03 18:04:33.844551: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-03 18:04:33.845661: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 18:04:33.846096: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 18:04:33.848872: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 18:04:33.851688: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 18:04:33.852181: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 18:04:33.855347: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 18:04:33.856448: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 18:04:33.860574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 18:04:33.861589: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 18:04:33.861669: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 18:04:33.862200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-03 18:04:33.862216: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-03 18:04:33.862225: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-03 18:04:33.863550: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2538 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-03 18:04:34.228585: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 18:04:34.228726: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 18:04:34.228748: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 18:04:34.228767: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 18:04:34.228786: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 18:04:34.228805: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 18:04:34.228823: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 18:04:34.228842: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 18:04:34.229839: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 18:04:34.231264: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 18:04:34.231333: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 18:04:34.231352: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 18:04:34.231370: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 18:04:34.231387: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 18:04:34.231405: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 18:04:34.231422: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 18:04:34.231439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 18:04:34.232387: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 18:04:34.232427: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-03 18:04:34.232435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-03 18:04:34.232442: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-03 18:04:34.233233: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2538 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-03 18:04:43.888143: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 18:04:44.172963: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 18:04:45.970316: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:45.970396: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:45.976851: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:45.976874: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:46.026324: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:46.026375: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:46.068112: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:46.068141: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.09GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:46.111080: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:46.111107: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.15GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-03 18:04:46.113098: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.48G (1589510144 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.430678: W tensorflow/core/common_runtime/bfc_allocator.cc:311] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature. 2025-02-03 18:04:46.473809: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.475027: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.476404: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.477232: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.487620: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.488204: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.488770: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.489318: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.489988: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.490610: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.504750: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.505304: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.567674: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.568253: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.590960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.591802: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.592619: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.593418: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.597833: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.598651: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.599499: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.600180: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.601723: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.601739: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-03 18:04:46.611404: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.611951: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.620849: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.621398: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.621950: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.622492: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.623072: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-03 18:04:46.623665: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 1.98G (2126381056 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 31 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 Detection mask done ! Trying to reset tf kernel 1015126 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1950 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3143 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.00028252601623535156 nb_pixel_total : 8221 time to create 1 rle with old method : 0.010663032531738281 length of segment : 110 time for calcul the mask position with numpy : 0.0007407665252685547 nb_pixel_total : 29594 time to create 1 rle with old method : 0.03486371040344238 length of segment : 254 time for calcul the mask position with numpy : 0.0018579959869384766 nb_pixel_total : 88221 time to create 1 rle with old method : 0.1139371395111084 length of segment : 378 time for calcul the mask position with numpy : 0.004532575607299805 nb_pixel_total : 222907 time to create 1 rle with new method : 0.011348962783813477 length of segment : 525 time for calcul the mask position with numpy : 0.004679679870605469 nb_pixel_total : 270900 time to create 1 rle with new method : 0.011380195617675781 length of segment : 607 time for calcul the mask position with numpy : 0.0014452934265136719 nb_pixel_total : 76386 time to create 1 rle with old method : 0.08470368385314941 length of segment : 398 time for calcul the mask position with numpy : 0.007317066192626953 nb_pixel_total : 400453 time to create 1 rle with new method : 0.020476818084716797 length of segment : 879 time for calcul the mask position with numpy : 0.0001327991485595703 nb_pixel_total : 2240 time to create 1 rle with old method : 0.002944469451904297 length of segment : 59 time for calcul the mask position with numpy : 7.176399230957031e-05 nb_pixel_total : 1807 time to create 1 rle with old method : 0.0021669864654541016 length of segment : 87 time for calcul the mask position with numpy : 0.003037691116333008 nb_pixel_total : 118499 time to create 1 rle with old method : 0.13164258003234863 length of segment : 742 time for calcul the mask position with numpy : 5.555152893066406e-05 nb_pixel_total : 945 time to create 1 rle with old method : 0.0010905265808105469 length of segment : 41 time for calcul the mask position with numpy : 0.0001590251922607422 nb_pixel_total : 6766 time to create 1 rle with old method : 0.008132696151733398 length of segment : 136 time for calcul the mask position with numpy : 0.0015459060668945312 nb_pixel_total : 105045 time to create 1 rle with old method : 0.11517477035522461 length of segment : 468 time for calcul the mask position with numpy : 0.00017762184143066406 nb_pixel_total : 6610 time to create 1 rle with old method : 0.007992029190063477 length of segment : 63 time for calcul the mask position with numpy : 0.0005278587341308594 nb_pixel_total : 24532 time to create 1 rle with old method : 0.043523311614990234 length of segment : 220 time for calcul the mask position with numpy : 9.560585021972656e-05 nb_pixel_total : 1540 time to create 1 rle with old method : 0.002481222152709961 length of segment : 55 time for calcul the mask position with numpy : 0.006412029266357422 nb_pixel_total : 375193 time to create 1 rle with new method : 0.017076969146728516 length of segment : 606 time for calcul the mask position with numpy : 0.0002503395080566406 nb_pixel_total : 7955 time to create 1 rle with old method : 0.009987831115722656 length of segment : 101 time for calcul the mask position with numpy : 0.0004508495330810547 nb_pixel_total : 20983 time to create 1 rle with old method : 0.02359938621520996 length of segment : 167 time for calcul the mask position with numpy : 0.0009562969207763672 nb_pixel_total : 20361 time to create 1 rle with old method : 0.029330730438232422 length of segment : 490 time for calcul the mask position with numpy : 0.001848459243774414 nb_pixel_total : 86710 time to create 1 rle with old method : 0.09623336791992188 length of segment : 406 time for calcul the mask position with numpy : 0.0008916854858398438 nb_pixel_total : 64992 time to create 1 rle with old method : 0.07080411911010742 length of segment : 359 time for calcul the mask position with numpy : 0.0010416507720947266 nb_pixel_total : 80078 time to create 1 rle with old method : 0.08913755416870117 length of segment : 414 time for calcul the mask position with numpy : 0.00033545494079589844 nb_pixel_total : 19542 time to create 1 rle with old method : 0.02179884910583496 length of segment : 211 time for calcul the mask position with numpy : 0.0022356510162353516 nb_pixel_total : 163725 time to create 1 rle with new method : 0.007005214691162109 length of segment : 392 time for calcul the mask position with numpy : 0.0012252330780029297 nb_pixel_total : 76997 time to create 1 rle with old method : 0.08365702629089355 length of segment : 409 time for calcul the mask position with numpy : 0.0009486675262451172 nb_pixel_total : 29017 time to create 1 rle with old method : 0.03545260429382324 length of segment : 288 time for calcul the mask position with numpy : 0.00012159347534179688 nb_pixel_total : 3131 time to create 1 rle with old method : 0.0039272308349609375 length of segment : 44 time for calcul the mask position with numpy : 9.441375732421875e-05 nb_pixel_total : 1870 time to create 1 rle with old method : 0.0024225711822509766 length of segment : 65 time for calcul the mask position with numpy : 9.059906005859375e-05 nb_pixel_total : 2252 time to create 1 rle with old method : 0.0028533935546875 length of segment : 77 time for calcul the mask position with numpy : 0.0003082752227783203 nb_pixel_total : 11274 time to create 1 rle with old method : 0.012939214706420898 length of segment : 200 time for calcul the mask position with numpy : 9.393692016601562e-05 nb_pixel_total : 2878 time to create 1 rle with old method : 0.0034859180450439453 length of segment : 58 time for calcul the mask position with numpy : 0.0024404525756835938 nb_pixel_total : 164204 time to create 1 rle with new method : 0.006604194641113281 length of segment : 400 time for calcul the mask position with numpy : 0.00022172927856445312 nb_pixel_total : 6298 time to create 1 rle with old method : 0.0076351165771484375 length of segment : 109 time for calcul the mask position with numpy : 0.0013179779052734375 nb_pixel_total : 88222 time to create 1 rle with old method : 0.10699629783630371 length of segment : 319 time for calcul the mask position with numpy : 0.0008175373077392578 nb_pixel_total : 52035 time to create 1 rle with old method : 0.0566716194152832 length of segment : 189 time for calcul the mask position with numpy : 0.002515077590942383 nb_pixel_total : 146388 time to create 1 rle with old method : 0.18558883666992188 length of segment : 457 time for calcul the mask position with numpy : 0.0020210742950439453 nb_pixel_total : 99790 time to create 1 rle with old method : 0.10695028305053711 length of segment : 443 time for calcul the mask position with numpy : 0.00022530555725097656 nb_pixel_total : 3333 time to create 1 rle with old method : 0.004149198532104492 length of segment : 181 time for calcul the mask position with numpy : 0.00133514404296875 nb_pixel_total : 87917 time to create 1 rle with old method : 0.0989677906036377 length of segment : 329 time for calcul the mask position with numpy : 0.0008687973022460938 nb_pixel_total : 70820 time to create 1 rle with old method : 0.07736349105834961 length of segment : 234 time for calcul the mask position with numpy : 0.0010287761688232422 nb_pixel_total : 67717 time to create 1 rle with old method : 0.07358956336975098 length of segment : 324 time for calcul the mask position with numpy : 0.0002913475036621094 nb_pixel_total : 13244 time to create 1 rle with old method : 0.015421867370605469 length of segment : 189 time for calcul the mask position with numpy : 9.965896606445312e-05 nb_pixel_total : 3667 time to create 1 rle with old method : 0.0044403076171875 length of segment : 62 time for calcul the mask position with numpy : 0.0006594657897949219 nb_pixel_total : 32715 time to create 1 rle with old method : 0.03725028038024902 length of segment : 187 time for calcul the mask position with numpy : 0.0002524852752685547 nb_pixel_total : 15887 time to create 1 rle with old method : 0.017690658569335938 length of segment : 151 time for calcul the mask position with numpy : 0.0004711151123046875 nb_pixel_total : 23036 time to create 1 rle with old method : 0.025425195693969727 length of segment : 185 time for calcul the mask position with numpy : 0.00018286705017089844 nb_pixel_total : 8275 time to create 1 rle with old method : 0.009405851364135742 length of segment : 134 time for calcul the mask position with numpy : 0.0002963542938232422 nb_pixel_total : 13901 time to create 1 rle with old method : 0.016479015350341797 length of segment : 123 time for calcul the mask position with numpy : 0.00011134147644042969 nb_pixel_total : 3731 time to create 1 rle with old method : 0.004670619964599609 length of segment : 100 time for calcul the mask position with numpy : 0.0005385875701904297 nb_pixel_total : 28448 time to create 1 rle with old method : 0.03370976448059082 length of segment : 263 time for calcul the mask position with numpy : 0.00022077560424804688 nb_pixel_total : 9104 time to create 1 rle with old method : 0.011322259902954102 length of segment : 195 time for calcul the mask position with numpy : 0.0010411739349365234 nb_pixel_total : 56663 time to create 1 rle with old method : 0.06256270408630371 length of segment : 439 time for calcul the mask position with numpy : 0.0012068748474121094 nb_pixel_total : 66138 time to create 1 rle with old method : 0.07155489921569824 length of segment : 358 time for calcul the mask position with numpy : 0.0050013065338134766 nb_pixel_total : 332679 time to create 1 rle with new method : 0.016463279724121094 length of segment : 870 time for calcul the mask position with numpy : 0.002451181411743164 nb_pixel_total : 66800 time to create 1 rle with old method : 0.07658791542053223 length of segment : 323 time for calcul the mask position with numpy : 0.00010752677917480469 nb_pixel_total : 1728 time to create 1 rle with old method : 0.0021708011627197266 length of segment : 68 time for calcul the mask position with numpy : 0.0045375823974609375 nb_pixel_total : 127638 time to create 1 rle with old method : 0.13804173469543457 length of segment : 557 time for calcul the mask position with numpy : 0.0009069442749023438 nb_pixel_total : 40590 time to create 1 rle with old method : 0.044187068939208984 length of segment : 403 time for calcul the mask position with numpy : 0.0003502368927001953 nb_pixel_total : 14754 time to create 1 rle with old method : 0.017843246459960938 length of segment : 104 time for calcul the mask position with numpy : 0.0018455982208251953 nb_pixel_total : 69458 time to create 1 rle with old method : 0.08144307136535645 length of segment : 226 time for calcul the mask position with numpy : 0.00042891502380371094 nb_pixel_total : 10913 time to create 1 rle with old method : 0.0133209228515625 length of segment : 168 time for calcul the mask position with numpy : 0.002260923385620117 nb_pixel_total : 67758 time to create 1 rle with old method : 0.07840895652770996 length of segment : 334 time for calcul the mask position with numpy : 0.0001957416534423828 nb_pixel_total : 1962 time to create 1 rle with old method : 0.00240325927734375 length of segment : 71 time for calcul the mask position with numpy : 0.0017418861389160156 nb_pixel_total : 48372 time to create 1 rle with old method : 0.056874752044677734 length of segment : 370 time for calcul the mask position with numpy : 0.0002567768096923828 nb_pixel_total : 3672 time to create 1 rle with old method : 0.004554033279418945 length of segment : 82 time for calcul the mask position with numpy : 0.005757808685302734 nb_pixel_total : 190702 time to create 1 rle with new method : 0.010880231857299805 length of segment : 586 time for calcul the mask position with numpy : 0.0008420944213867188 nb_pixel_total : 15207 time to create 1 rle with old method : 0.032037973403930664 length of segment : 151 time for calcul the mask position with numpy : 0.0017428398132324219 nb_pixel_total : 30216 time to create 1 rle with old method : 0.0377659797668457 length of segment : 283 time for calcul the mask position with numpy : 0.0018954277038574219 nb_pixel_total : 59807 time to create 1 rle with old method : 0.06679606437683105 length of segment : 315 time for calcul the mask position with numpy : 0.0005583763122558594 nb_pixel_total : 9923 time to create 1 rle with old method : 0.011408567428588867 length of segment : 118 time for calcul the mask position with numpy : 0.0019402503967285156 nb_pixel_total : 71526 time to create 1 rle with old method : 0.07581138610839844 length of segment : 339 time for calcul the mask position with numpy : 0.0002796649932861328 nb_pixel_total : 3732 time to create 1 rle with old method : 0.00444340705871582 length of segment : 100 time for calcul the mask position with numpy : 0.0010364055633544922 nb_pixel_total : 31075 time to create 1 rle with old method : 0.03497886657714844 length of segment : 182 time for calcul the mask position with numpy : 0.00010085105895996094 nb_pixel_total : 4179 time to create 1 rle with old method : 0.005264759063720703 length of segment : 70 time for calcul the mask position with numpy : 0.0014967918395996094 nb_pixel_total : 63528 time to create 1 rle with old method : 0.07283711433410645 length of segment : 402 time for calcul the mask position with numpy : 0.0018818378448486328 nb_pixel_total : 45871 time to create 1 rle with old method : 0.053640127182006836 length of segment : 447 time for calcul the mask position with numpy : 0.0005788803100585938 nb_pixel_total : 26298 time to create 1 rle with old method : 0.028575420379638672 length of segment : 282 time for calcul the mask position with numpy : 0.0003662109375 nb_pixel_total : 16360 time to create 1 rle with old method : 0.018944501876831055 length of segment : 216 time for calcul the mask position with numpy : 0.002148151397705078 nb_pixel_total : 48529 time to create 1 rle with old method : 0.05232834815979004 length of segment : 429 time for calcul the mask position with numpy : 0.0028810501098632812 nb_pixel_total : 76301 time to create 1 rle with old method : 0.08425784111022949 length of segment : 365 time for calcul the mask position with numpy : 0.0003180503845214844 nb_pixel_total : 7046 time to create 1 rle with old method : 0.008280038833618164 length of segment : 127 time for calcul the mask position with numpy : 0.0037992000579833984 nb_pixel_total : 115236 time to create 1 rle with old method : 0.13316130638122559 length of segment : 538 time for calcul the mask position with numpy : 0.0002493858337402344 nb_pixel_total : 2227 time to create 1 rle with old method : 0.0026836395263671875 length of segment : 104 time for calcul the mask position with numpy : 0.0013535022735595703 nb_pixel_total : 69421 time to create 1 rle with old method : 0.07482051849365234 length of segment : 381 time for calcul the mask position with numpy : 0.00020074844360351562 nb_pixel_total : 3714 time to create 1 rle with old method : 0.006161689758300781 length of segment : 61 time for calcul the mask position with numpy : 0.002059459686279297 nb_pixel_total : 71214 time to create 1 rle with old method : 0.08861565589904785 length of segment : 235 time for calcul the mask position with numpy : 0.0019965171813964844 nb_pixel_total : 67259 time to create 1 rle with old method : 0.07644462585449219 length of segment : 321 time for calcul the mask position with numpy : 0.0005350112915039062 nb_pixel_total : 13419 time to create 1 rle with old method : 0.01521611213684082 length of segment : 185 time for calcul the mask position with numpy : 0.0008993148803710938 nb_pixel_total : 31246 time to create 1 rle with old method : 0.03562808036804199 length of segment : 186 time for calcul the mask position with numpy : 0.0005462169647216797 nb_pixel_total : 14778 time to create 1 rle with old method : 0.01810598373413086 length of segment : 183 time for calcul the mask position with numpy : 0.00024390220642089844 nb_pixel_total : 3522 time to create 1 rle with old method : 0.0041348934173583984 length of segment : 99 time for calcul the mask position with numpy : 0.00019669532775878906 nb_pixel_total : 3992 time to create 1 rle with old method : 0.004891633987426758 length of segment : 67 time for calcul the mask position with numpy : 0.0026743412017822266 nb_pixel_total : 104078 time to create 1 rle with old method : 0.10931110382080078 length of segment : 313 time for calcul the mask position with numpy : 0.00012993812561035156 nb_pixel_total : 1686 time to create 1 rle with old method : 0.001972675323486328 length of segment : 67 time for calcul the mask position with numpy : 0.0022547245025634766 nb_pixel_total : 64672 time to create 1 rle with old method : 0.06967902183532715 length of segment : 330 time for calcul the mask position with numpy : 0.0006973743438720703 nb_pixel_total : 10894 time to create 1 rle with old method : 0.017513751983642578 length of segment : 162 time for calcul the mask position with numpy : 0.0006389617919921875 nb_pixel_total : 12683 time to create 1 rle with old method : 0.014303207397460938 length of segment : 168 time for calcul the mask position with numpy : 0.0011191368103027344 nb_pixel_total : 59938 time to create 1 rle with old method : 0.06614089012145996 length of segment : 398 time for calcul the mask position with numpy : 0.004101276397705078 nb_pixel_total : 149402 time to create 1 rle with old method : 0.16068243980407715 length of segment : 708 time for calcul the mask position with numpy : 0.0005970001220703125 nb_pixel_total : 4665 time to create 1 rle with old method : 0.005942821502685547 length of segment : 105 time for calcul the mask position with numpy : 0.0003936290740966797 nb_pixel_total : 13149 time to create 1 rle with old method : 0.01593494415283203 length of segment : 100 time for calcul the mask position with numpy : 0.0016658306121826172 nb_pixel_total : 42458 time to create 1 rle with old method : 0.046532630920410156 length of segment : 615 time for calcul the mask position with numpy : 0.0013382434844970703 nb_pixel_total : 69348 time to create 1 rle with old method : 0.07456278800964355 length of segment : 329 time for calcul the mask position with numpy : 0.0010066032409667969 nb_pixel_total : 52811 time to create 1 rle with old method : 0.06159687042236328 length of segment : 447 time for calcul the mask position with numpy : 0.0019257068634033203 nb_pixel_total : 80392 time to create 1 rle with old method : 0.08785295486450195 length of segment : 326 time for calcul the mask position with numpy : 0.0004336833953857422 nb_pixel_total : 14425 time to create 1 rle with old method : 0.017443180084228516 length of segment : 116 time for calcul the mask position with numpy : 0.0011835098266601562 nb_pixel_total : 33272 time to create 1 rle with old method : 0.0398557186126709 length of segment : 379 time for calcul the mask position with numpy : 0.0048825740814208984 nb_pixel_total : 146665 time to create 1 rle with old method : 0.15601348876953125 length of segment : 723 time for calcul the mask position with numpy : 0.004719734191894531 nb_pixel_total : 149894 time to create 1 rle with old method : 0.16591310501098633 length of segment : 405 time for calcul the mask position with numpy : 0.00045013427734375 nb_pixel_total : 11263 time to create 1 rle with old method : 0.01470184326171875 length of segment : 111 time for calcul the mask position with numpy : 0.001890420913696289 nb_pixel_total : 64296 time to create 1 rle with old method : 0.0707547664642334 length of segment : 446 time for calcul the mask position with numpy : 0.0003154277801513672 nb_pixel_total : 5434 time to create 1 rle with old method : 0.006161212921142578 length of segment : 128 time for calcul the mask position with numpy : 0.0012054443359375 nb_pixel_total : 39880 time to create 1 rle with old method : 0.04589438438415527 length of segment : 394 time for calcul the mask position with numpy : 0.0017480850219726562 nb_pixel_total : 78085 time to create 1 rle with old method : 0.08411550521850586 length of segment : 261 time for calcul the mask position with numpy : 0.0015285015106201172 nb_pixel_total : 52777 time to create 1 rle with old method : 0.06208300590515137 length of segment : 220 time for calcul the mask position with numpy : 0.00015306472778320312 nb_pixel_total : 1675 time to create 1 rle with old method : 0.0031838417053222656 length of segment : 39 time for calcul the mask position with numpy : 0.0005881786346435547 nb_pixel_total : 7219 time to create 1 rle with old method : 0.01324605941772461 length of segment : 228 time for calcul the mask position with numpy : 0.002225160598754883 nb_pixel_total : 58587 time to create 1 rle with old method : 0.09106779098510742 length of segment : 295 time for calcul the mask position with numpy : 0.0027396678924560547 nb_pixel_total : 92837 time to create 1 rle with old method : 0.10326552391052246 length of segment : 389 time for calcul the mask position with numpy : 0.002126455307006836 nb_pixel_total : 71667 time to create 1 rle with old method : 0.10654735565185547 length of segment : 577 time for calcul the mask position with numpy : 0.0019996166229248047 nb_pixel_total : 79516 time to create 1 rle with old method : 0.12534165382385254 length of segment : 378 time for calcul the mask position with numpy : 0.0014636516571044922 nb_pixel_total : 46453 time to create 1 rle with old method : 0.05321526527404785 length of segment : 315 time for calcul the mask position with numpy : 0.003022432327270508 nb_pixel_total : 113551 time to create 1 rle with old method : 0.1253039836883545 length of segment : 527 time for calcul the mask position with numpy : 0.0009522438049316406 nb_pixel_total : 21827 time to create 1 rle with old method : 0.0251309871673584 length of segment : 346 time for calcul the mask position with numpy : 0.0004723072052001953 nb_pixel_total : 12901 time to create 1 rle with old method : 0.01574254035949707 length of segment : 126 time for calcul the mask position with numpy : 0.0012392997741699219 nb_pixel_total : 35046 time to create 1 rle with old method : 0.040151357650756836 length of segment : 201 time for calcul the mask position with numpy : 0.0034792423248291016 nb_pixel_total : 137625 time to create 1 rle with old method : 0.15284323692321777 length of segment : 442 time for calcul the mask position with numpy : 0.00020503997802734375 nb_pixel_total : 3067 time to create 1 rle with old method : 0.003853321075439453 length of segment : 59 time for calcul the mask position with numpy : 0.0013725757598876953 nb_pixel_total : 34656 time to create 1 rle with old method : 0.040627479553222656 length of segment : 302 time for calcul the mask position with numpy : 0.0008611679077148438 nb_pixel_total : 35364 time to create 1 rle with old method : 0.039433956146240234 length of segment : 326 time for calcul the mask position with numpy : 0.005752086639404297 nb_pixel_total : 161601 time to create 1 rle with new method : 0.01274871826171875 length of segment : 707 time for calcul the mask position with numpy : 0.0009682178497314453 nb_pixel_total : 25136 time to create 1 rle with old method : 0.02988600730895996 length of segment : 193 time for calcul the mask position with numpy : 0.0007255077362060547 nb_pixel_total : 5298 time to create 1 rle with old method : 0.006573915481567383 length of segment : 224 time for calcul the mask position with numpy : 0.003498077392578125 nb_pixel_total : 102028 time to create 1 rle with old method : 0.10903382301330566 length of segment : 416 time for calcul the mask position with numpy : 0.0007631778717041016 nb_pixel_total : 16721 time to create 1 rle with old method : 0.027120590209960938 length of segment : 158 time for calcul the mask position with numpy : 0.0011987686157226562 nb_pixel_total : 69362 time to create 1 rle with old method : 0.07487082481384277 length of segment : 295 time for calcul the mask position with numpy : 0.0007584095001220703 nb_pixel_total : 20453 time to create 1 rle with old method : 0.02186751365661621 length of segment : 166 time for calcul the mask position with numpy : 0.0004322528839111328 nb_pixel_total : 14588 time to create 1 rle with old method : 0.016124725341796875 length of segment : 160 time for calcul the mask position with numpy : 0.0013091564178466797 nb_pixel_total : 77407 time to create 1 rle with old method : 0.08332276344299316 length of segment : 507 time for calcul the mask position with numpy : 0.0009906291961669922 nb_pixel_total : 32231 time to create 1 rle with old method : 0.03591299057006836 length of segment : 226 time for calcul the mask position with numpy : 0.0005786418914794922 nb_pixel_total : 12578 time to create 1 rle with old method : 0.014793634414672852 length of segment : 157 time for calcul the mask position with numpy : 0.005901336669921875 nb_pixel_total : 173642 time to create 1 rle with new method : 0.01188349723815918 length of segment : 608 time for calcul the mask position with numpy : 0.0009953975677490234 nb_pixel_total : 22667 time to create 1 rle with old method : 0.026551008224487305 length of segment : 174 time for calcul the mask position with numpy : 0.00042247772216796875 nb_pixel_total : 8736 time to create 1 rle with old method : 0.009972333908081055 length of segment : 195 time for calcul the mask position with numpy : 0.031243324279785156 nb_pixel_total : 973214 time to create 1 rle with new method : 0.06572556495666504 length of segment : 2677 time for calcul the mask position with numpy : 0.0006067752838134766 nb_pixel_total : 17794 time to create 1 rle with old method : 0.019385576248168945 length of segment : 188 time for calcul the mask position with numpy : 0.0006053447723388672 nb_pixel_total : 12496 time to create 1 rle with old method : 0.01386404037475586 length of segment : 209 time for calcul the mask position with numpy : 0.0035271644592285156 nb_pixel_total : 154286 time to create 1 rle with new method : 0.007265329360961914 length of segment : 503 time for calcul the mask position with numpy : 0.0005729198455810547 nb_pixel_total : 11155 time to create 1 rle with old method : 0.01296091079711914 length of segment : 187 time for calcul the mask position with numpy : 0.010646581649780273 nb_pixel_total : 463289 time to create 1 rle with new method : 0.016492843627929688 length of segment : 837 time for calcul the mask position with numpy : 0.008988618850708008 nb_pixel_total : 475616 time to create 1 rle with new method : 0.018040895462036133 length of segment : 851 time for calcul the mask position with numpy : 0.00019407272338867188 nb_pixel_total : 3951 time to create 1 rle with old method : 0.0045855045318603516 length of segment : 85 time for calcul the mask position with numpy : 0.00030732154846191406 nb_pixel_total : 4521 time to create 1 rle with old method : 0.005215167999267578 length of segment : 112 time for calcul the mask position with numpy : 0.006457328796386719 nb_pixel_total : 454732 time to create 1 rle with new method : 0.015476226806640625 length of segment : 1385 time for calcul the mask position with numpy : 0.00015783309936523438 nb_pixel_total : 1364 time to create 1 rle with old method : 0.001683950424194336 length of segment : 62 time for calcul the mask position with numpy : 0.005333662033081055 nb_pixel_total : 448138 time to create 1 rle with new method : 0.010861635208129883 length of segment : 974 time for calcul the mask position with numpy : 0.0007061958312988281 nb_pixel_total : 34911 time to create 1 rle with old method : 0.039452552795410156 length of segment : 433 time for calcul the mask position with numpy : 0.0002760887145996094 nb_pixel_total : 2999 time to create 1 rle with old method : 0.003528118133544922 length of segment : 83 time for calcul the mask position with numpy : 0.0005612373352050781 nb_pixel_total : 12810 time to create 1 rle with old method : 0.013925313949584961 length of segment : 106 time for calcul the mask position with numpy : 0.0008587837219238281 nb_pixel_total : 20539 time to create 1 rle with old method : 0.02314901351928711 length of segment : 205 time for calcul the mask position with numpy : 0.0002560615539550781 nb_pixel_total : 3821 time to create 1 rle with old method : 0.004654645919799805 length of segment : 52 time for calcul the mask position with numpy : 0.0016918182373046875 nb_pixel_total : 50183 time to create 1 rle with old method : 0.05290699005126953 length of segment : 263 time for calcul the mask position with numpy : 0.00015091896057128906 nb_pixel_total : 1500 time to create 1 rle with old method : 0.001874685287475586 length of segment : 60 time for calcul the mask position with numpy : 0.0020318031311035156 nb_pixel_total : 70787 time to create 1 rle with old method : 0.07956171035766602 length of segment : 288 time for calcul the mask position with numpy : 0.00044274330139160156 nb_pixel_total : 5952 time to create 1 rle with old method : 0.007055044174194336 length of segment : 149 time for calcul the mask position with numpy : 0.0032684803009033203 nb_pixel_total : 89217 time to create 1 rle with old method : 0.09918069839477539 length of segment : 771 time for calcul the mask position with numpy : 0.0009598731994628906 nb_pixel_total : 23104 time to create 1 rle with old method : 0.026821374893188477 length of segment : 185 time for calcul the mask position with numpy : 0.0014295578002929688 nb_pixel_total : 40304 time to create 1 rle with old method : 0.04455137252807617 length of segment : 222 time for calcul the mask position with numpy : 0.0005693435668945312 nb_pixel_total : 12696 time to create 1 rle with old method : 0.01460576057434082 length of segment : 156 time for calcul the mask position with numpy : 0.00016498565673828125 nb_pixel_total : 2333 time to create 1 rle with old method : 0.0028083324432373047 length of segment : 53 time for calcul the mask position with numpy : 0.0030236244201660156 nb_pixel_total : 117045 time to create 1 rle with old method : 0.1219027042388916 length of segment : 465 time for calcul the mask position with numpy : 0.0005779266357421875 nb_pixel_total : 27022 time to create 1 rle with old method : 0.030722856521606445 length of segment : 295 time for calcul the mask position with numpy : 0.0003094673156738281 nb_pixel_total : 5276 time to create 1 rle with old method : 0.006228446960449219 length of segment : 98 time for calcul the mask position with numpy : 0.0013446807861328125 nb_pixel_total : 6916 time to create 1 rle with old method : 0.008454561233520508 length of segment : 163 time for calcul the mask position with numpy : 0.0008306503295898438 nb_pixel_total : 50606 time to create 1 rle with old method : 0.05363798141479492 length of segment : 274 time for calcul the mask position with numpy : 0.001142740249633789 nb_pixel_total : 28439 time to create 1 rle with old method : 0.03196144104003906 length of segment : 235 time for calcul the mask position with numpy : 0.0005326271057128906 nb_pixel_total : 7951 time to create 1 rle with old method : 0.009102821350097656 length of segment : 139 time for calcul the mask position with numpy : 0.0018668174743652344 nb_pixel_total : 61447 time to create 1 rle with old method : 0.07279467582702637 length of segment : 398 time for calcul the mask position with numpy : 0.0006806850433349609 nb_pixel_total : 15883 time to create 1 rle with old method : 0.019524335861206055 length of segment : 188 time for calcul the mask position with numpy : 0.0005857944488525391 nb_pixel_total : 15199 time to create 1 rle with old method : 0.01666879653930664 length of segment : 177 time for calcul the mask position with numpy : 0.002326488494873047 nb_pixel_total : 77870 time to create 1 rle with old method : 0.08216547966003418 length of segment : 333 time for calcul the mask position with numpy : 0.0009632110595703125 nb_pixel_total : 23072 time to create 1 rle with old method : 0.026833534240722656 length of segment : 307 time for calcul the mask position with numpy : 0.0009911060333251953 nb_pixel_total : 28337 time to create 1 rle with old method : 0.03136849403381348 length of segment : 403 time for calcul the mask position with numpy : 0.0034689903259277344 nb_pixel_total : 163306 time to create 1 rle with new method : 0.004650115966796875 length of segment : 537 time for calcul the mask position with numpy : 0.0005288124084472656 nb_pixel_total : 13900 time to create 1 rle with old method : 0.015138387680053711 length of segment : 201 time for calcul the mask position with numpy : 0.00046443939208984375 nb_pixel_total : 12901 time to create 1 rle with old method : 0.01562356948852539 length of segment : 126 time for calcul the mask position with numpy : 0.0005002021789550781 nb_pixel_total : 14150 time to create 1 rle with old method : 0.016466617584228516 length of segment : 127 time for calcul the mask position with numpy : 0.0001723766326904297 nb_pixel_total : 4968 time to create 1 rle with old method : 0.0061457157135009766 length of segment : 155 time for calcul the mask position with numpy : 0.0005888938903808594 nb_pixel_total : 19799 time to create 1 rle with old method : 0.02149033546447754 length of segment : 190 time for calcul the mask position with numpy : 0.0011627674102783203 nb_pixel_total : 23290 time to create 1 rle with old method : 0.02553272247314453 length of segment : 167 time for calcul the mask position with numpy : 0.0014619827270507812 nb_pixel_total : 65351 time to create 1 rle with old method : 0.07370853424072266 length of segment : 351 time for calcul the mask position with numpy : 0.0006349086761474609 nb_pixel_total : 19347 time to create 1 rle with old method : 0.02048182487487793 length of segment : 212 time for calcul the mask position with numpy : 0.00031375885009765625 nb_pixel_total : 8020 time to create 1 rle with old method : 0.00897836685180664 length of segment : 116 time for calcul the mask position with numpy : 0.00016164779663085938 nb_pixel_total : 1694 time to create 1 rle with old method : 0.0020570755004882812 length of segment : 45 time for calcul the mask position with numpy : 0.0022711753845214844 nb_pixel_total : 108006 time to create 1 rle with old method : 0.11319541931152344 length of segment : 394 time for calcul the mask position with numpy : 0.011432170867919922 nb_pixel_total : 420798 time to create 1 rle with new method : 0.018861770629882812 length of segment : 1086 time spent for convertir_results : 17.277445316314697 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 210 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 59293 save missing photos in datou_result : time spend for datou_step_exec : 62.80684018135071 time spend to save output : 3.2423973083496094 total time spend for step 1 : 66.04923748970032 step2:crop_condition Mon Feb 3 18:05:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 210 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 17 About to insert : list_path_to_insert length 17 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 18 About to insert : list_path_to_insert length 18 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 45 About to insert : list_path_to_insert length 45 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1333240473, 1333240469, 1333234645, 1333234627, 1333234617, 1333234610, 1333234606, 1333234598, 1333234551, 1333234548] Looping around the photos to save general results len do output : 112 /-3646997655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997672Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997697Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997712Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3648348385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997714Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3651960187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3651960190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997798Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997835Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997698Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997735Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997731Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997728Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997732Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3651960188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997750Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997772Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997833Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997677Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3651960185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997668Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997661Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997673Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997670Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997709Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997706Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3651960186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997718Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997730Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997723Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997720Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3648348384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3651960189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3651960191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997794Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997793Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997782Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997788Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997789Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997780Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997797Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997817Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997675Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997783Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997795Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997838Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3648348383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997674Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997734Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997846Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997707Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997790Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997791Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997792Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3646997786Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240473', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240469', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234645', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234627', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234617', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234610', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234606', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234598', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234551', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234548', None, None, None, None, None, '2539822') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 346 time used for this insertion : 0.035819292068481445 save_final save missing photos in datou_result : time spend for datou_step_exec : 40.20604681968689 time spend to save output : 0.03900504112243652 total time spend for step 2 : 40.245051860809326 step3:thcl Mon Feb 3 18:06:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.010903596878051758 time to convert the images to numpy array : 0.03772258758544922 time to import caffe and check if the image exist : 0.020494937896728516 time to convert the images to numpy array : 0.08938837051391602 time to import caffe and check if the image exist : 0.014630317687988281 time to convert the images to numpy array : 0.09920167922973633 time to import caffe and check if the image exist : 0.01728343963623047 time to convert the images to numpy array : 0.10317015647888184 time to import caffe and check if the image exist : 0.02062368392944336 time to convert the images to numpy array : 0.10905170440673828 time to import caffe and check if the image exist : 0.02025151252746582 time to convert the images to numpy array : 0.12336254119873047 time to import caffe and check if the image exist : 0.020429372787475586 time to convert the images to numpy array : 0.13047051429748535 time to import caffe and check if the image exist : 0.017056941986083984 time to convert the images to numpy array : 0.13466143608093262 time to import caffe and check if the image exist : 0.019296646118164062 time to convert the images to numpy array : 0.14163875579833984 time to import caffe and check if the image exist : 0.016123056411743164 time to convert the images to numpy array : 0.1460285186767578 total time to convert the images to numpy array : 0.5078756809234619 list photo_ids error: [] list photo_ids correct : [-3646997790, -3646997791, -3646997792, -3646997786, -3646997842, -3646997675, -3646997783, -3646997795, -3646997810, -3646997820, -3646997818, -3646997840, -3646997838, -3648348383, -3646997674, -3646997710, -3646997782, -3646997788, -3646997789, -3646997780, -3646997800, -3646997797, -3646997822, -3646997825, -3646997817, -3646997813, -3646997827, -3646997834, -3646997655, -3646997682, -3646997672, -3646997692, -3646997697, -3646997693, -3646997712, -3648348385, -3646997714, -3646997738, -3651960187, -3646997763, -3651960190, -3646997757, -3646997777, -3646997798, -3646997835, -3646997698, -3646997704, -3646997735, -3646997722, -3646997731, -3646997728, -3646997732, -3646997749, -3651960188, -3646997750, -3646997773, -3646997772, -3646997806, -3646997815, -3646997833, -3646997841, -3646997844, -3646997843, -3646997677, -3648348384, -3646997754, -3646997745, -3651960189, -3646997759, -3646997758, -3646997770, -3651960191, -3646997771, -3646997768, -3646997794, -3646997793, -3646997687, -3646997670, -3646997689, -3646997678, -3646997709, -3646997706, -3651960186, -3646997718, -3646997721, -3646997730, -3646997723, -3646997720, -3651960185, -3646997688, -3646997764, -3646997826, -3646997654, -3646997666, -3646997668, -3646997664, -3646997661, -3646997673, -3646997671, -3646997676, -3646997727, -3646997734, -3646997755, -3646997751, -3646997760, -3646997762, -3646997774, -3646997778, -3646997808, -3646997846, -3646997686, -3646997707] number of photos to traite : 112 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3364 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3145 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0266973972320557 time used to do the prediction : 0.29206395149230957 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3143 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3143 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.4897971153259277 time used to do the prediction : 0.29309678077697754 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.7881393432617188e-05 save missing photos in datou_result : time spend for datou_step_exec : 10.87684941291809 time spend to save output : 0.00070953369140625 total time spend for step 3 : 10.877558946609497 step4:merge_mask_thcl_custom Mon Feb 3 18:06:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 210 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1333240473, 1333240469, 1333234645, 1333234627, 1333234617, 1333234610, 1333234606, 1333234598, 1333234551, 1333234548] Looping around the photos to save general results len do output : 10 /1333240473Didn't retrieve data .Didn't retrieve data . /1333240469Didn't retrieve data .Didn't retrieve data . /1333234645Didn't retrieve data .Didn't retrieve data . /1333234627Didn't retrieve data .Didn't retrieve data . /1333234617Didn't retrieve data .Didn't retrieve data . /1333234610Didn't retrieve data .Didn't retrieve data . /1333234606Didn't retrieve data .Didn't retrieve data . /1333234598Didn't retrieve data .Didn't retrieve data . /1333234551Didn't retrieve data .Didn't retrieve data . /1333234548Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240473', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240469', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234645', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234627', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234617', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234610', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234606', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234598', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234551', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234548', None, None, None, None, None, '2539822') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013149738311767578 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.07044744491577148 time spend to save output : 0.013610601425170898 total time spend for step 4 : 0.08405804634094238 step5:rle_unique_nms_with_priority Mon Feb 3 18:06:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 10.944386005401611 time for calcul the mask position with numpy : 0.4235267639160156 nb_pixel_total : 5311805 time to create 1 rle with new method : 1.053431749343872 time for calcul the mask position with numpy : 0.023488283157348633 nb_pixel_total : 118367 time to create 1 rle with old method : 0.13190770149230957 time for calcul the mask position with numpy : 0.0251467227935791 nb_pixel_total : 374236 time to create 1 rle with new method : 0.7384705543518066 time for calcul the mask position with numpy : 0.020798206329345703 nb_pixel_total : 763 time to create 1 rle with old method : 0.0009169578552246094 time for calcul the mask position with numpy : 0.02090740203857422 nb_pixel_total : 6544 time to create 1 rle with old method : 0.007369041442871094 time for calcul the mask position with numpy : 0.021566152572631836 nb_pixel_total : 24408 time to create 1 rle with old method : 0.027017593383789062 time for calcul the mask position with numpy : 0.02060723304748535 nb_pixel_total : 6661 time to create 1 rle with old method : 0.007748603820800781 time for calcul the mask position with numpy : 0.02144622802734375 nb_pixel_total : 941 time to create 1 rle with old method : 0.0011157989501953125 time for calcul the mask position with numpy : 0.02060699462890625 nb_pixel_total : 1801 time to create 1 rle with old method : 0.0021021366119384766 time for calcul the mask position with numpy : 0.0218660831451416 nb_pixel_total : 88003 time to create 1 rle with old method : 0.09932327270507812 time for calcul the mask position with numpy : 0.0245361328125 nb_pixel_total : 222097 time to create 1 rle with new method : 0.8782868385314941 time for calcul the mask position with numpy : 0.03417706489562988 nb_pixel_total : 2218 time to create 1 rle with old method : 0.002551555633544922 time for calcul the mask position with numpy : 0.03612661361694336 nb_pixel_total : 398306 time to create 1 rle with new method : 0.4226346015930176 time for calcul the mask position with numpy : 0.04032564163208008 nb_pixel_total : 270126 time to create 1 rle with new method : 0.426332950592041 time for calcul the mask position with numpy : 0.03503012657165527 nb_pixel_total : 75543 time to create 1 rle with old method : 0.07781481742858887 time for calcul the mask position with numpy : 0.03383946418762207 nb_pixel_total : 29433 time to create 1 rle with old method : 0.030861616134643555 time for calcul the mask position with numpy : 0.03185462951660156 nb_pixel_total : 8178 time to create 1 rle with old method : 0.009280204772949219 time for calcul the mask position with numpy : 0.034464120864868164 nb_pixel_total : 110810 time to create 1 rle with old method : 0.13120579719543457 create new chi : 5.072956085205078 time to delete rle : 0.01035165786743164 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 13271 TO DO : save crop sub photo not yet done ! save time : 0.7156424522399902 nb_obj : 21 nb_hashtags : 8 time to prepare the origin masks : 9.868950366973877 time for calcul the mask position with numpy : 0.3644847869873047 nb_pixel_total : 6237103 time to create 1 rle with new method : 0.5419857501983643 time for calcul the mask position with numpy : 0.019978046417236328 nb_pixel_total : 20813 time to create 1 rle with old method : 0.021392345428466797 time for calcul the mask position with numpy : 0.01956033706665039 nb_pixel_total : 13152 time to create 1 rle with old method : 0.014645576477050781 time for calcul the mask position with numpy : 0.020174741744995117 nb_pixel_total : 3951 time to create 1 rle with old method : 0.004838228225708008 time for calcul the mask position with numpy : 0.02054572105407715 nb_pixel_total : 36111 time to create 1 rle with old method : 0.0382537841796875 time for calcul the mask position with numpy : 0.021207809448242188 nb_pixel_total : 2536 time to create 1 rle with old method : 0.0029189586639404297 time for calcul the mask position with numpy : 0.02172088623046875 nb_pixel_total : 86398 time to create 1 rle with old method : 0.0933527946472168 time for calcul the mask position with numpy : 0.0207366943359375 nb_pixel_total : 19994 time to create 1 rle with old method : 0.02094125747680664 time for calcul the mask position with numpy : 0.0251009464263916 nb_pixel_total : 13030 time to create 1 rle with old method : 0.014139413833618164 time for calcul the mask position with numpy : 0.035436153411865234 nb_pixel_total : 6208 time to create 1 rle with old method : 0.006864309310913086 time for calcul the mask position with numpy : 0.03394484519958496 nb_pixel_total : 28866 time to create 1 rle with old method : 0.04448580741882324 time for calcul the mask position with numpy : 0.035158395767211914 nb_pixel_total : 163171 time to create 1 rle with new method : 1.2643015384674072 time for calcul the mask position with numpy : 0.03400611877441406 nb_pixel_total : 79390 time to create 1 rle with old method : 0.08344864845275879 time for calcul the mask position with numpy : 0.03754544258117676 nb_pixel_total : 2862 time to create 1 rle with old method : 0.0032269954681396484 time for calcul the mask position with numpy : 0.03220057487487793 nb_pixel_total : 2247 time to create 1 rle with old method : 0.00250244140625 time for calcul the mask position with numpy : 0.03282451629638672 nb_pixel_total : 1856 time to create 1 rle with old method : 0.002017498016357422 time for calcul the mask position with numpy : 0.03361797332763672 nb_pixel_total : 19497 time to create 1 rle with old method : 0.019704818725585938 time for calcul the mask position with numpy : 0.03237271308898926 nb_pixel_total : 88031 time to create 1 rle with old method : 0.09159135818481445 time for calcul the mask position with numpy : 0.03589224815368652 nb_pixel_total : 3108 time to create 1 rle with old method : 0.005011558532714844 time for calcul the mask position with numpy : 0.04252791404724121 nb_pixel_total : 145838 time to create 1 rle with old method : 0.1594865322113037 time for calcul the mask position with numpy : 0.024899959564208984 nb_pixel_total : 11224 time to create 1 rle with old method : 0.012323856353759766 time for calcul the mask position with numpy : 0.02100062370300293 nb_pixel_total : 64854 time to create 1 rle with old method : 0.06727910041809082 create new chi : 3.526127338409424 time to delete rle : 0.0015954971313476562 batch 1 Loaded 25 chid ids of type : 3726 Number RLEs to save : 12381 TO DO : save crop sub photo not yet done ! save time : 0.6813087463378906 nb_obj : 18 nb_hashtags : 5 time to prepare the origin masks : 9.128275632858276 time for calcul the mask position with numpy : 0.13208484649658203 nb_pixel_total : 6154637 time to create 1 rle with new method : 0.7137875556945801 time for calcul the mask position with numpy : 0.022891521453857422 nb_pixel_total : 66660 time to create 1 rle with old method : 0.09592199325561523 time for calcul the mask position with numpy : 0.021066665649414062 nb_pixel_total : 70688 time to create 1 rle with old method : 0.07347965240478516 time for calcul the mask position with numpy : 0.020375728607177734 nb_pixel_total : 1724 time to create 1 rle with old method : 0.0019333362579345703 time for calcul the mask position with numpy : 0.020313501358032227 nb_pixel_total : 28340 time to create 1 rle with old method : 0.029804706573486328 time for calcul the mask position with numpy : 0.020513534545898438 nb_pixel_total : 57555 time to create 1 rle with old method : 0.05942487716674805 time for calcul the mask position with numpy : 0.02095651626586914 nb_pixel_total : 19815 time to create 1 rle with old method : 0.021408557891845703 time for calcul the mask position with numpy : 0.019937515258789062 nb_pixel_total : 9058 time to create 1 rle with old method : 0.010037899017333984 time for calcul the mask position with numpy : 0.020009994506835938 nb_pixel_total : 3718 time to create 1 rle with old method : 0.004149436950683594 time for calcul the mask position with numpy : 0.03235602378845215 nb_pixel_total : 22955 time to create 1 rle with old method : 0.024619579315185547 time for calcul the mask position with numpy : 0.03366875648498535 nb_pixel_total : 13684 time to create 1 rle with old method : 0.014209508895874023 time for calcul the mask position with numpy : 0.024774551391601562 nb_pixel_total : 127460 time to create 1 rle with old method : 0.1470935344696045 time for calcul the mask position with numpy : 0.022667407989501953 nb_pixel_total : 8239 time to create 1 rle with old method : 0.011219263076782227 time for calcul the mask position with numpy : 0.020061254501342773 nb_pixel_total : 67591 time to create 1 rle with old method : 0.06775903701782227 time for calcul the mask position with numpy : 0.02016592025756836 nb_pixel_total : 32593 time to create 1 rle with old method : 0.036583900451660156 time for calcul the mask position with numpy : 0.019542932510375977 nb_pixel_total : 3653 time to create 1 rle with old method : 0.0038170814514160156 time for calcul the mask position with numpy : 0.020102977752685547 nb_pixel_total : 15845 time to create 1 rle with old method : 0.015526771545410156 time for calcul the mask position with numpy : 0.01992654800415039 nb_pixel_total : 13197 time to create 1 rle with old method : 0.013680458068847656 time for calcul the mask position with numpy : 0.022350072860717773 nb_pixel_total : 332828 time to create 1 rle with new method : 0.25275373458862305 create new chi : 2.180711030960083 time to delete rle : 0.001027822494506836 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 11415 TO DO : save crop sub photo not yet done ! save time : 0.6348521709442139 nb_obj : 25 nb_hashtags : 5 time to prepare the origin masks : 10.395799160003662 time for calcul the mask position with numpy : 0.4295797348022461 nb_pixel_total : 6147719 time to create 1 rle with new method : 0.6185646057128906 time for calcul the mask position with numpy : 0.020460128784179688 nb_pixel_total : 76181 time to create 1 rle with old method : 0.07752299308776855 time for calcul the mask position with numpy : 0.020568370819091797 nb_pixel_total : 190335 time to create 1 rle with new method : 0.26279282569885254 time for calcul the mask position with numpy : 0.020172834396362305 nb_pixel_total : 30141 time to create 1 rle with old method : 0.031180143356323242 time for calcul the mask position with numpy : 0.019743919372558594 nb_pixel_total : 48310 time to create 1 rle with old method : 0.05031633377075195 time for calcul the mask position with numpy : 0.0190122127532959 nb_pixel_total : 22764 time to create 1 rle with old method : 0.02428436279296875 time for calcul the mask position with numpy : 0.02027297019958496 nb_pixel_total : 115190 time to create 1 rle with old method : 0.12179994583129883 time for calcul the mask position with numpy : 0.02033090591430664 nb_pixel_total : 3706 time to create 1 rle with old method : 0.004210710525512695 time for calcul the mask position with numpy : 0.02220296859741211 nb_pixel_total : 2216 time to create 1 rle with old method : 0.002438783645629883 time for calcul the mask position with numpy : 0.03513312339782715 nb_pixel_total : 6975 time to create 1 rle with old method : 0.007865428924560547 time for calcul the mask position with numpy : 0.03386378288269043 nb_pixel_total : 4151 time to create 1 rle with old method : 0.004767656326293945 time for calcul the mask position with numpy : 0.034375905990600586 nb_pixel_total : 69287 time to create 1 rle with old method : 0.07339358329772949 time for calcul the mask position with numpy : 0.02930760383605957 nb_pixel_total : 30956 time to create 1 rle with old method : 0.03344154357910156 time for calcul the mask position with numpy : 0.021243572235107422 nb_pixel_total : 3730 time to create 1 rle with old method : 0.004064083099365234 time for calcul the mask position with numpy : 0.021463394165039062 nb_pixel_total : 15143 time to create 1 rle with old method : 0.0167238712310791 time for calcul the mask position with numpy : 0.020857810974121094 nb_pixel_total : 10659 time to create 1 rle with old method : 0.01199030876159668 time for calcul the mask position with numpy : 0.02086663246154785 nb_pixel_total : 2784 time to create 1 rle with old method : 0.003011465072631836 time for calcul the mask position with numpy : 0.020198583602905273 nb_pixel_total : 4208 time to create 1 rle with old method : 0.0046672821044921875 time for calcul the mask position with numpy : 0.020990610122680664 nb_pixel_total : 67529 time to create 1 rle with old method : 0.06910181045532227 time for calcul the mask position with numpy : 0.021471023559570312 nb_pixel_total : 3663 time to create 1 rle with old method : 0.004157066345214844 time for calcul the mask position with numpy : 0.02087879180908203 nb_pixel_total : 1956 time to create 1 rle with old method : 0.0021028518676757812 time for calcul the mask position with numpy : 0.020619869232177734 nb_pixel_total : 6959 time to create 1 rle with old method : 0.0075151920318603516 time for calcul the mask position with numpy : 0.021020174026489258 nb_pixel_total : 65962 time to create 1 rle with old method : 0.06656193733215332 time for calcul the mask position with numpy : 0.02129364013671875 nb_pixel_total : 48554 time to create 1 rle with old method : 0.049066781997680664 time for calcul the mask position with numpy : 0.019844770431518555 nb_pixel_total : 59681 time to create 1 rle with old method : 0.06161808967590332 time for calcul the mask position with numpy : 0.01999688148498535 nb_pixel_total : 11481 time to create 1 rle with old method : 0.012010335922241211 create new chi : 2.670441150665283 time to delete rle : 0.0012059211730957031 batch 1 Loaded 29 chid ids of type : 3726 Number RLEs to save : 13818 TO DO : save crop sub photo not yet done ! save time : 0.7104432582855225 nb_obj : 16 nb_hashtags : 4 time to prepare the origin masks : 7.400420427322388 time for calcul the mask position with numpy : 0.1833653450012207 nb_pixel_total : 6519803 time to create 1 rle with new method : 0.3482356071472168 time for calcul the mask position with numpy : 0.023325443267822266 nb_pixel_total : 148951 time to create 1 rle with old method : 0.1622781753540039 time for calcul the mask position with numpy : 0.02037334442138672 nb_pixel_total : 1677 time to create 1 rle with old method : 0.0019638538360595703 time for calcul the mask position with numpy : 0.020694732666015625 nb_pixel_total : 70082 time to create 1 rle with old method : 0.09314417839050293 time for calcul the mask position with numpy : 0.02160501480102539 nb_pixel_total : 46027 time to create 1 rle with old method : 0.05013585090637207 time for calcul the mask position with numpy : 0.019406557083129883 nb_pixel_total : 2448 time to create 1 rle with old method : 0.002942323684692383 time for calcul the mask position with numpy : 0.019816160202026367 nb_pixel_total : 3983 time to create 1 rle with old method : 0.004335880279541016 time for calcul the mask position with numpy : 0.019842147827148438 nb_pixel_total : 3509 time to create 1 rle with old method : 0.003707408905029297 time for calcul the mask position with numpy : 0.019948959350585938 nb_pixel_total : 67146 time to create 1 rle with old method : 0.06922769546508789 time for calcul the mask position with numpy : 0.02237415313720703 nb_pixel_total : 15079 time to create 1 rle with old method : 0.016705751419067383 time for calcul the mask position with numpy : 0.03370046615600586 nb_pixel_total : 59224 time to create 1 rle with old method : 0.06145620346069336 time for calcul the mask position with numpy : 0.03363513946533203 nb_pixel_total : 4653 time to create 1 rle with old method : 0.00510859489440918 time for calcul the mask position with numpy : 0.034612178802490234 nb_pixel_total : 11134 time to create 1 rle with old method : 0.012331247329711914 time for calcul the mask position with numpy : 0.03297019004821777 nb_pixel_total : 31109 time to create 1 rle with old method : 0.03301048278808594 time for calcul the mask position with numpy : 0.035456180572509766 nb_pixel_total : 39408 time to create 1 rle with old method : 0.042595863342285156 time for calcul the mask position with numpy : 0.0322728157043457 nb_pixel_total : 12597 time to create 1 rle with old method : 0.013162851333618164 time for calcul the mask position with numpy : 0.03125286102294922 nb_pixel_total : 13410 time to create 1 rle with old method : 0.014514684677124023 create new chi : 1.5775349140167236 time to delete rle : 0.0009610652923583984 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 9893 TO DO : save crop sub photo not yet done ! save time : 2.029344320297241 nb_obj : 19 nb_hashtags : 7 time to prepare the origin masks : 8.738186836242676 time for calcul the mask position with numpy : 0.284160852432251 nb_pixel_total : 5984766 time to create 1 rle with new method : 1.0499706268310547 time for calcul the mask position with numpy : 0.02150893211364746 nb_pixel_total : 52572 time to create 1 rle with old method : 0.054924964904785156 time for calcul the mask position with numpy : 0.020660400390625 nb_pixel_total : 33260 time to create 1 rle with old method : 0.03461122512817383 time for calcul the mask position with numpy : 0.020689964294433594 nb_pixel_total : 14377 time to create 1 rle with old method : 0.014866352081298828 time for calcul the mask position with numpy : 0.020875930786132812 nb_pixel_total : 58516 time to create 1 rle with old method : 0.06142783164978027 time for calcul the mask position with numpy : 0.021448850631713867 nb_pixel_total : 145582 time to create 1 rle with old method : 0.14961910247802734 time for calcul the mask position with numpy : 0.022684812545776367 nb_pixel_total : 113565 time to create 1 rle with old method : 0.11959624290466309 time for calcul the mask position with numpy : 0.020764589309692383 nb_pixel_total : 5406 time to create 1 rle with old method : 0.006118297576904297 time for calcul the mask position with numpy : 0.0211179256439209 nb_pixel_total : 80248 time to create 1 rle with old method : 0.08367085456848145 time for calcul the mask position with numpy : 0.02514815330505371 nb_pixel_total : 7204 time to create 1 rle with old method : 0.007275104522705078 time for calcul the mask position with numpy : 0.03275275230407715 nb_pixel_total : 1667 time to create 1 rle with old method : 0.0018427371978759766 time for calcul the mask position with numpy : 0.033704280853271484 nb_pixel_total : 77841 time to create 1 rle with old method : 0.08125877380371094 time for calcul the mask position with numpy : 0.0322115421295166 nb_pixel_total : 63262 time to create 1 rle with old method : 0.0650014877319336 time for calcul the mask position with numpy : 0.030531883239746094 nb_pixel_total : 40518 time to create 1 rle with old method : 0.042334794998168945 time for calcul the mask position with numpy : 0.0201871395111084 nb_pixel_total : 46840 time to create 1 rle with old method : 0.047455787658691406 time for calcul the mask position with numpy : 0.020580530166625977 nb_pixel_total : 11218 time to create 1 rle with old method : 0.011972188949584961 time for calcul the mask position with numpy : 0.023276090621948242 nb_pixel_total : 149235 time to create 1 rle with old method : 0.15662503242492676 time for calcul the mask position with numpy : 0.03267312049865723 nb_pixel_total : 71253 time to create 1 rle with old method : 0.0738976001739502 time for calcul the mask position with numpy : 0.034073591232299805 nb_pixel_total : 13474 time to create 1 rle with old method : 0.014992713928222656 time for calcul the mask position with numpy : 0.035247087478637695 nb_pixel_total : 79436 time to create 1 rle with old method : 0.08384084701538086 create new chi : 2.9705653190612793 time to delete rle : 0.0013496875762939453 batch 1 Loaded 23 chid ids of type : 3726 Number RLEs to save : 14317 TO DO : save crop sub photo not yet done ! save time : 0.7823352813720703 nb_obj : 19 nb_hashtags : 9 time to prepare the origin masks : 8.867242813110352 time for calcul the mask position with numpy : 0.3825409412384033 nb_pixel_total : 6126511 time to create 1 rle with new method : 1.003727912902832 time for calcul the mask position with numpy : 0.019411802291870117 nb_pixel_total : 20371 time to create 1 rle with old method : 0.02014899253845215 time for calcul the mask position with numpy : 0.019834041595458984 nb_pixel_total : 164527 time to create 1 rle with new method : 0.9931437969207764 time for calcul the mask position with numpy : 0.01915574073791504 nb_pixel_total : 12569 time to create 1 rle with old method : 0.012790441513061523 time for calcul the mask position with numpy : 0.019109249114990234 nb_pixel_total : 34951 time to create 1 rle with old method : 0.03590822219848633 time for calcul the mask position with numpy : 0.01931142807006836 nb_pixel_total : 22573 time to create 1 rle with old method : 0.023024559020996094 time for calcul the mask position with numpy : 0.020743370056152344 nb_pixel_total : 16654 time to create 1 rle with old method : 0.01739215850830078 time for calcul the mask position with numpy : 0.019565820693969727 nb_pixel_total : 25077 time to create 1 rle with old method : 0.024888277053833008 time for calcul the mask position with numpy : 0.020706892013549805 nb_pixel_total : 101921 time to create 1 rle with old method : 0.10261321067810059 time for calcul the mask position with numpy : 0.019199132919311523 nb_pixel_total : 5254 time to create 1 rle with old method : 0.005534648895263672 time for calcul the mask position with numpy : 0.020012617111206055 nb_pixel_total : 14522 time to create 1 rle with old method : 0.015349864959716797 time for calcul the mask position with numpy : 0.02035069465637207 nb_pixel_total : 34916 time to create 1 rle with old method : 0.035700082778930664 time for calcul the mask position with numpy : 0.020241975784301758 nb_pixel_total : 32160 time to create 1 rle with old method : 0.03168225288391113 time for calcul the mask position with numpy : 0.020139694213867188 nb_pixel_total : 172942 time to create 1 rle with new method : 0.3312208652496338 time for calcul the mask position with numpy : 0.027190208435058594 nb_pixel_total : 137512 time to create 1 rle with old method : 0.14936137199401855 time for calcul the mask position with numpy : 0.03911137580871582 nb_pixel_total : 23 time to create 1 rle with old method : 0.00010228157043457031 time for calcul the mask position with numpy : 0.037622928619384766 nb_pixel_total : 3060 time to create 1 rle with old method : 0.003766298294067383 time for calcul the mask position with numpy : 0.03300762176513672 nb_pixel_total : 12872 time to create 1 rle with old method : 0.015722274780273438 time for calcul the mask position with numpy : 0.032181501388549805 nb_pixel_total : 34521 time to create 1 rle with old method : 0.036060333251953125 time for calcul the mask position with numpy : 0.03029322624206543 nb_pixel_total : 77304 time to create 1 rle with old method : 0.07815742492675781 create new chi : 3.8641107082366943 time to delete rle : 0.0010828971862792969 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 12423 TO DO : save crop sub photo not yet done ! save time : 0.690544605255127 nb_obj : 16 nb_hashtags : 5 time to prepare the origin masks : 6.886790037155151 time for calcul the mask position with numpy : 0.08278608322143555 nb_pixel_total : 4686299 time to create 1 rle with new method : 0.9092435836791992 time for calcul the mask position with numpy : 0.020047664642333984 nb_pixel_total : 12480 time to create 1 rle with old method : 0.013455390930175781 time for calcul the mask position with numpy : 0.020743370056152344 nb_pixel_total : 82122 time to create 1 rle with old method : 0.08772468566894531 time for calcul the mask position with numpy : 0.020279407501220703 nb_pixel_total : 1350 time to create 1 rle with old method : 0.0016927719116210938 time for calcul the mask position with numpy : 0.02020406723022461 nb_pixel_total : 4556 time to create 1 rle with old method : 0.005246400833129883 time for calcul the mask position with numpy : 0.020914316177368164 nb_pixel_total : 921 time to create 1 rle with old method : 0.0015478134155273438 time for calcul the mask position with numpy : 0.02067089080810547 nb_pixel_total : 89795 time to create 1 rle with old method : 0.09639573097229004 time for calcul the mask position with numpy : 0.02093362808227539 nb_pixel_total : 3943 time to create 1 rle with old method : 0.0055904388427734375 time for calcul the mask position with numpy : 0.022860050201416016 nb_pixel_total : 11083 time to create 1 rle with old method : 0.01786637306213379 time for calcul the mask position with numpy : 0.026221036911010742 nb_pixel_total : 491950 time to create 1 rle with new method : 0.6213173866271973 time for calcul the mask position with numpy : 0.024343490600585938 nb_pixel_total : 462730 time to create 1 rle with new method : 0.6164565086364746 time for calcul the mask position with numpy : 0.021007776260375977 nb_pixel_total : 34877 time to create 1 rle with old method : 0.03718972206115723 time for calcul the mask position with numpy : 0.023209333419799805 nb_pixel_total : 154248 time to create 1 rle with new method : 0.44759511947631836 time for calcul the mask position with numpy : 0.020038127899169922 nb_pixel_total : 9359 time to create 1 rle with old method : 0.011120319366455078 time for calcul the mask position with numpy : 0.028177261352539062 nb_pixel_total : 978025 time to create 1 rle with new method : 0.3393583297729492 time for calcul the mask position with numpy : 0.020828723907470703 nb_pixel_total : 17782 time to create 1 rle with old method : 0.019171476364135742 time for calcul the mask position with numpy : 0.020866870880126953 nb_pixel_total : 8720 time to create 1 rle with old method : 0.009129762649536133 create new chi : 3.775695562362671 time to delete rle : 0.002051830291748047 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 19022 TO DO : save crop sub photo not yet done ! save time : 1.015840768814087 nb_obj : 22 nb_hashtags : 5 time to prepare the origin masks : 10.89436936378479 time for calcul the mask position with numpy : 0.15111231803894043 nb_pixel_total : 6428150 time to create 1 rle with new method : 1.25502347946167 time for calcul the mask position with numpy : 0.02617621421813965 nb_pixel_total : 77685 time to create 1 rle with old method : 0.1368093490600586 time for calcul the mask position with numpy : 0.02693915367126465 nb_pixel_total : 70573 time to create 1 rle with old method : 0.10913491249084473 time for calcul the mask position with numpy : 0.022105693817138672 nb_pixel_total : 28341 time to create 1 rle with old method : 0.03171563148498535 time for calcul the mask position with numpy : 0.022326231002807617 nb_pixel_total : 23006 time to create 1 rle with old method : 0.02545785903930664 time for calcul the mask position with numpy : 0.020163536071777344 nb_pixel_total : 15790 time to create 1 rle with old method : 0.016719341278076172 time for calcul the mask position with numpy : 0.020186424255371094 nb_pixel_total : 11857 time to create 1 rle with old method : 0.01307225227355957 time for calcul the mask position with numpy : 0.020634889602661133 nb_pixel_total : 5243 time to create 1 rle with old method : 0.005717039108276367 time for calcul the mask position with numpy : 0.019809961318969727 nb_pixel_total : 1553 time to create 1 rle with old method : 0.0019381046295166016 time for calcul the mask position with numpy : 0.020415306091308594 nb_pixel_total : 50169 time to create 1 rle with old method : 0.07069945335388184 time for calcul the mask position with numpy : 0.02439570426940918 nb_pixel_total : 2318 time to create 1 rle with old method : 0.003747701644897461 time for calcul the mask position with numpy : 0.025784730911254883 nb_pixel_total : 116769 time to create 1 rle with old method : 0.12555766105651855 time for calcul the mask position with numpy : 0.022110700607299805 nb_pixel_total : 2769 time to create 1 rle with old method : 0.003286600112915039 time for calcul the mask position with numpy : 0.02764272689819336 nb_pixel_total : 40034 time to create 1 rle with old method : 0.04311800003051758 time for calcul the mask position with numpy : 0.038599252700805664 nb_pixel_total : 5929 time to create 1 rle with old method : 0.006829261779785156 time for calcul the mask position with numpy : 0.0368809700012207 nb_pixel_total : 92239 time to create 1 rle with old method : 0.09707403182983398 time for calcul the mask position with numpy : 0.03498673439025879 nb_pixel_total : 1491 time to create 1 rle with old method : 0.0017070770263671875 time for calcul the mask position with numpy : 0.038977622985839844 nb_pixel_total : 29562 time to create 1 rle with old method : 0.03086400032043457 time for calcul the mask position with numpy : 0.03225970268249512 nb_pixel_total : 3783 time to create 1 rle with old method : 0.004166841506958008 time for calcul the mask position with numpy : 0.021334409713745117 nb_pixel_total : 12689 time to create 1 rle with old method : 0.013679742813110352 time for calcul the mask position with numpy : 0.021002769470214844 nb_pixel_total : 6862 time to create 1 rle with old method : 0.00767207145690918 time for calcul the mask position with numpy : 0.019834518432617188 nb_pixel_total : 20446 time to create 1 rle with old method : 0.021439313888549805 time for calcul the mask position with numpy : 0.020608901977539062 nb_pixel_total : 2982 time to create 1 rle with old method : 0.0031671524047851562 create new chi : 2.7894601821899414 time to delete rle : 0.0009872913360595703 batch 1 Loaded 24 chid ids of type : 3726 Number RLEs to save : 11540 TO DO : save crop sub photo not yet done ! save time : 0.7135715484619141 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 4.145772695541382 time for calcul the mask position with numpy : 0.3801000118255615 nb_pixel_total : 6596675 time to create 1 rle with new method : 1.207047462463379 time for calcul the mask position with numpy : 0.020444154739379883 nb_pixel_total : 65262 time to create 1 rle with old method : 0.06807184219360352 time for calcul the mask position with numpy : 0.021000146865844727 nb_pixel_total : 23084 time to create 1 rle with old method : 0.025938034057617188 time for calcul the mask position with numpy : 0.02030467987060547 nb_pixel_total : 19314 time to create 1 rle with old method : 0.020183086395263672 time for calcul the mask position with numpy : 0.020936965942382812 nb_pixel_total : 163114 time to create 1 rle with new method : 0.5476129055023193 time for calcul the mask position with numpy : 0.020432472229003906 nb_pixel_total : 7972 time to create 1 rle with old method : 0.008807897567749023 time for calcul the mask position with numpy : 0.019809484481811523 nb_pixel_total : 1675 time to create 1 rle with old method : 0.0019371509552001953 time for calcul the mask position with numpy : 0.020040512084960938 nb_pixel_total : 19725 time to create 1 rle with old method : 0.02060699462890625 time for calcul the mask position with numpy : 0.020812034606933594 nb_pixel_total : 107693 time to create 1 rle with old method : 0.10928916931152344 time for calcul the mask position with numpy : 0.03347325325012207 nb_pixel_total : 4921 time to create 1 rle with old method : 0.00532078742980957 time for calcul the mask position with numpy : 0.033046722412109375 nb_pixel_total : 12828 time to create 1 rle with old method : 0.014182806015014648 time for calcul the mask position with numpy : 0.021221160888671875 nb_pixel_total : 14106 time to create 1 rle with old method : 0.01463770866394043 time for calcul the mask position with numpy : 0.02121591567993164 nb_pixel_total : 13871 time to create 1 rle with old method : 0.014993906021118164 create new chi : 2.7600667476654053 time to delete rle : 0.0007610321044921875 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 7422 TO DO : save crop sub photo not yet done ! save time : 0.3926224708557129 map_output_result : {1333240473: (0.0, 'Should be the crop_list due to order', 0.0), 1333240469: (0.0, 'Should be the crop_list due to order', 0.0), 1333234645: (0.0, 'Should be the crop_list due to order', 0.0), 1333234627: (0.0, 'Should be the crop_list due to order', 0.0), 1333234617: (0.0, 'Should be the crop_list due to order', 0.0), 1333234610: (0.0, 'Should be the crop_list due to order', 0.0), 1333234606: (0.0, 'Should be the crop_list due to order', 0.0), 1333234598: (0.0, 'Should be the crop_list due to order', 0.0), 1333234551: (0.0, 'Should be the crop_list due to order', 0.0), 1333234548: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1333240473, 1333240469, 1333234645, 1333234627, 1333234617, 1333234610, 1333234606, 1333234598, 1333234551, 1333234548] Looping around the photos to save general results len do output : 10 /1333240473.Didn't retrieve data . /1333240469.Didn't retrieve data . /1333234645.Didn't retrieve data . /1333234627.Didn't retrieve data . /1333234617.Didn't retrieve data . /1333234610.Didn't retrieve data . /1333234606.Didn't retrieve data . /1333234598.Didn't retrieve data . /1333234551.Didn't retrieve data . /1333234548.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240473', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240469', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234645', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234627', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234617', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234610', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234606', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234598', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234551', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234548', None, None, None, None, None, '2539822') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.012343645095825195 save_final save missing photos in datou_result : time spend for datou_step_exec : 127.72553658485413 time spend to save output : 0.012789249420166016 total time spend for step 5 : 127.73832583427429 step6:crop_condition Mon Feb 3 18:08:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 215 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 4869462 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738602518_1014818 we have uploaded 11 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.1631734371185303 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738602524_1014818 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8432133197784424 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738602529_1014818 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.6258494853973389 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738602533_1014818 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9232196807861328 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 4869462 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738602537_1014818 we have uploaded 12 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.220090389251709 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738602542_1014818 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.2037277221679688 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738602545_1014818 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0174834728240967 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1333240473, 1333240469, 1333234645, 1333234627, 1333234617, 1333234610, 1333234606, 1333234598, 1333234551, 1333234548] Looping around the photos to save general results len do output : 42 /1334280877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280882Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280886Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280891Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280892Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280894Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280898Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280912Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334280930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240473', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240469', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234645', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234627', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234617', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234610', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234606', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234598', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234551', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234548', None, None, None, None, None, '2539822') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 136 time used for this insertion : 0.03307199478149414 save_final save missing photos in datou_result : time spend for datou_step_exec : 32.25177001953125 time spend to save output : 0.034242868423461914 total time spend for step 6 : 32.28601288795471 step7:ventilate_hashtags_in_portfolio Mon Feb 3 18:09:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20176483 get user id for portfolio 20176483 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176483 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','Carton_brun','kraft','Carton_gris','metal','flou','environnement','autre_refus','papier','mal_croppe','Teint_Dans_La_Masse','plastique')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176483 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','Carton_brun','kraft','Carton_gris','metal','flou','environnement','autre_refus','papier','mal_croppe','Teint_Dans_La_Masse','plastique')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20185284,20185285,20185286,20185287,20185288,20185289,20185290,20185291,20185292,20185293,20185294,20185295?tags=environnement,mal_croppe,Carton_brun,flou,autre_refus,Teint_Dans_La_Masse,cartonnette,metal,plastique,Carton_gris,kraft,papier Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1333240473, 1333240469, 1333234645, 1333234627, 1333234617, 1333234610, 1333234606, 1333234598, 1333234551, 1333234548] Looping around the photos to save general results len do output : 1 /20176483. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240473', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240469', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234645', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234627', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234617', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234610', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234606', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234598', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234551', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234548', None, None, None, None, None, '2539822') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014059066772460938 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.4385461807250977 time spend to save output : 0.01436161994934082 total time spend for step 7 : 1.4529078006744385 step8:final Mon Feb 3 18:09:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1333240473: ('0.084060949697339',), 1333240469: ('0.084060949697339',), 1333234645: ('0.084060949697339',), 1333234627: ('0.084060949697339',), 1333234617: ('0.084060949697339',), 1333234610: ('0.084060949697339',), 1333234606: ('0.084060949697339',), 1333234598: ('0.084060949697339',), 1333234551: ('0.084060949697339',), 1333234548: ('0.084060949697339',)} new output for save of step final : {1333240473: ('0.084060949697339',), 1333240469: ('0.084060949697339',), 1333234645: ('0.084060949697339',), 1333234627: ('0.084060949697339',), 1333234617: ('0.084060949697339',), 1333234610: ('0.084060949697339',), 1333234606: ('0.084060949697339',), 1333234598: ('0.084060949697339',), 1333234551: ('0.084060949697339',), 1333234548: ('0.084060949697339',)} [1333240473, 1333240469, 1333234645, 1333234627, 1333234617, 1333234610, 1333234606, 1333234598, 1333234551, 1333234548] Looping around the photos to save general results len do output : 10 /1333240473.Didn't retrieve data . /1333240469.Didn't retrieve data . /1333234645.Didn't retrieve data . /1333234627.Didn't retrieve data . /1333234617.Didn't retrieve data . /1333234610.Didn't retrieve data . /1333234606.Didn't retrieve data . /1333234598.Didn't retrieve data . /1333234551.Didn't retrieve data . /1333234548.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240473', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240469', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234645', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234627', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234617', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234610', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234606', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234598', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234551', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234548', None, None, None, None, None, '2539822') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013345479965209961 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10611367225646973 time spend to save output : 0.013733148574829102 total time spend for step 8 : 0.11984682083129883 step9:velours_tree Mon Feb 3 18:09:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.02957630157470703 time spend to save output : 2.956390380859375e-05 total time spend for step 9 : 0.029605865478515625 step10:send_mail_cod Mon Feb 3 18:09:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20176483_03-02-2025_18_09_08.pdf 20185285 imagette201852851738602548 20185287 imagette201852871738602548 20185288 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201852881738602548 20185290 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201852901738602550 20185291 imagette201852911738602552 20185292 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201852921738602552 20185294 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201852941738602552 20185286 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201852861738602553 20185289 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201852891738602554 20185293 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201852931738602556 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20176483 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20185284,20185285,20185286,20185287,20185288,20185289,20185290,20185291,20185292,20185293,20185294,20185295?tags=environnement,mal_croppe,Carton_brun,flou,autre_refus,Teint_Dans_La_Masse,cartonnette,metal,plastique,Carton_gris,kraft,papier your option no_mail is active, we will not send the real mail to your client args[1333240473] : ('0.084060949697339',) no score found for photo 1333240473 We are sending mail with results at report@fotonower.com args[1333240469] : ('0.084060949697339',) no score found for photo 1333240469 We are sending mail with results at report@fotonower.com args[1333234645] : ('0.084060949697339',) no score found for photo 1333234645 We are sending mail with results at report@fotonower.com args[1333234627] : ('0.084060949697339',) no score found for photo 1333234627 We are sending mail with results at report@fotonower.com args[1333234617] : ('0.084060949697339',) no score found for photo 1333234617 We are sending mail with results at report@fotonower.com args[1333234610] : ('0.084060949697339',) no score found for photo 1333234610 We are sending mail with results at report@fotonower.com args[1333234606] : ('0.084060949697339',) no score found for photo 1333234606 We are sending mail with results at report@fotonower.com args[1333234598] : ('0.084060949697339',) no score found for photo 1333234598 We are sending mail with results at report@fotonower.com args[1333234551] : ('0.084060949697339',) no score found for photo 1333234551 We are sending mail with results at report@fotonower.com args[1333234548] : ('0.084060949697339',) no score found for photo 1333234548 We are sending mail with results at report@fotonower.com refus_total : 0.084060949697339 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20176483 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1333234432,1333495938,1333493424,1333493404,1333493380,1333493255,1333493249,1333493244,1333240473,1333240469,1333234645,1333234627,1333234617,1333234610,1333234551,1333234548,1333234544,1333234539,1333234428,1333234606) Found this number of photos: 20 begin to download photo : 1333234432 begin to download photo : 1333493255 begin to download photo : 1333234645 begin to download photo : 1333234548 download finish for photo 1333234548 begin to download photo : 1333234544 download finish for photo 1333234645 begin to download photo : 1333234627 download finish for photo 1333234432 begin to download photo : 1333495938 download finish for photo 1333493255 begin to download photo : 1333493249 download finish for photo 1333234544 begin to download photo : 1333234539 download finish for photo 1333234627 begin to download photo : 1333234617 download finish for photo 1333493249 begin to download photo : 1333493244 download finish for photo 1333495938 begin to download photo : 1333493424 download finish for photo 1333493244 begin to download photo : 1333240473 download finish for photo 1333234617 begin to download photo : 1333234610 download finish for photo 1333234539 begin to download photo : 1333234428 download finish for photo 1333493424 begin to download photo : 1333493404 download finish for photo 1333234428 begin to download photo : 1333234606 download finish for photo 1333240473 begin to download photo : 1333240469 download finish for photo 1333234610 begin to download photo : 1333234551 download finish for photo 1333493404 begin to download photo : 1333493380 download finish for photo 1333234606 download finish for photo 1333234551 download finish for photo 1333493380 download finish for photo 1333240469 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176483_03-02-2025_18_09_08.pdf results_Auto_P20176483_03-02-2025_18_09_08.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176483_03-02-2025_18_09_08.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20176483','results_Auto_P20176483_03-02-2025_18_09_08.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176483_03-02-2025_18_09_08.pdf','pdf','','1.35','0.084060949697339') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1333240473, 1333240469, 1333234645, 1333234627, 1333234617, 1333234610, 1333234606, 1333234598, 1333234551, 1333234548] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240473', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240469', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234645', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234627', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234617', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234610', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234606', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234598', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234551', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234548', None, None, None, None, None, '2539822') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.012690544128417969 save_final save missing photos in datou_result : time spend for datou_step_exec : 13.225878477096558 time spend to save output : 0.012925386428833008 total time spend for step 10 : 13.23880386352539 step11:split_time_score Mon Feb 3 18:09:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 28),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 31012025 20176483 Nombre de photos uploadées : 28 / 23040 (0%) 31012025 20176483 Nombre de photos taguées (types de déchets): 0 / 28 (0%) 31012025 20176483 Nombre de photos taguées (volume) : 0 / 28 (0%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 8.344650268554688e-06 ???????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0015482902526855469 elapsed_time : insert_dashboard_record_day_entry 0.022330522537231445 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.07948558805196004 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128150_01-02-2025_04_30_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128150 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128150 AND mptpi.`type`=4207 To do Qualite : 0.04288956511275865 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128151_01-02-2025_03_54_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128151 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128151 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128152 order by id desc limit 1 Qualite : 0.08437942123195745 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176483_03-02-2025_18_09_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176483 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176483 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176484 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176485 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176486 order by id desc limit 1 Qualite : 0.18765544939683718 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176487_03-02-2025_14_04_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176487 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176487 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176512 order by id desc limit 1 Qualite : 0.17372436966684815 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176513_03-02-2025_13_11_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176513 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176513 AND mptpi.`type`=3594 To do Qualite : 0.1374952317954594 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176514_03-02-2025_15_09_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176514 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176514 AND mptpi.`type`=3327 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176515_03-02-2025_14_56_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176515 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176516 order by id desc limit 1 Qualite : 0.20109579764144703 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176517_03-02-2025_13_00_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176517 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176517 AND mptpi.`type`=3594 To do Qualite : 0.21289584725935826 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176518_03-02-2025_12_51_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176518 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176518 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176519 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176520 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20116177 order by id desc limit 1 Qualite : 0.11710340391431627 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176521_03-02-2025_14_35_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176521 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176521 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176522 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176523 order by id desc limit 1 Qualite : 0.06999912744886748 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176524_03-02-2025_12_55_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176524 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176524 AND mptpi.`type`=4207 To do Qualite : 0.045846082848016355 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20176525_03-02-2025_13_05_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20176525 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20176525 AND mptpi.`type`=4207 To do Qualite : 0.08687190479353087 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20135985_01-02-2025_06_07_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20135985 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20135985 AND mptpi.`type`=4207 To do Qualite : 0.11540364153479511 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20135986_01-02-2025_07_01_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20135986 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11512 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11521 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11520 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11516 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11516 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11519 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11519 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11513 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11513 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11577 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11577 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11515 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11514 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11514 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11523 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11515 doesn't seem to be define in the database( WARNING : type of input 3 of step 11514 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11512 doesn't seem to be define in the database( WARNING : type of input 2 of step 11516 doesn't seem to be define in the database( WARNING : output 1 of step 11512 have datatype=2 whereas input 1 of step 11519 have datatype=7 WARNING : type of output 2 of step 11519 doesn't seem to be define in the database( WARNING : type of input 1 of step 11513 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11515 have datatype=10 whereas input 3 of step 11522 have datatype=6 WARNING : type of input 2 of step 11577 doesn't seem to be define in the database( WARNING : output 1 of step 11513 have datatype=7 whereas input 2 of step 11577 have datatype=None WARNING : type of output 3 of step 11577 doesn't seem to be define in the database( WARNING : type of input 1 of step 11515 doesn't seem to be define in the database( WARNING : output 0 of step 11515 have datatype=10 whereas input 0 of step 11583 have datatype=18 WARNING : type of input 5 of step 11522 doesn't seem to be define in the database( WARNING : output 0 of step 11583 have datatype=11 whereas input 5 of step 11522 have datatype=None WARNING : type of output 1 of step 11521 doesn't seem to be define in the database( WARNING : type of input 3 of step 11516 doesn't seem to be define in the database( WARNING : type of output 1 of step 11520 doesn't seem to be define in the database( WARNING : type of input 3 of step 11516 doesn't seem to be define in the database( WARNING : output 0 of step 11519 have datatype=1 whereas input 0 of step 11513 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20135986 AND mptpi.`type`=4203 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20135988_01-02-2025_06_46_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20135988 order by id desc limit 1 Qualite : 0.22578661033761485 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20135989 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20135989 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20135990_01-02-2025_05_51_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20135990 order by id desc limit 1 Qualite : 0.20874959339445653 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128161_01-02-2025_01_24_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128161 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128161 AND mptpi.`type`=3594 To do Qualite : 0.280106560658458 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128162_01-02-2025_04_44_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128162 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128162 AND mptpi.`type`=4200 To do Qualite : 0.198585640541933 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128894_01-02-2025_07_08_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128894 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128894 AND mptpi.`type`=3327 To do Qualite : 0.08584882097122022 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128895_01-02-2025_04_15_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128895 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128895 AND mptpi.`type`=4207 To do Qualite : 0.021879504220965417 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128165_01-02-2025_01_28_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128165 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128165 AND mptpi.`type`=4207 To do Qualite : 0.04651606487101653 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128166_01-02-2025_01_54_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128166 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128166 AND mptpi.`type`=4207 To do Qualite : 0.03317004296775434 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128167_01-02-2025_01_48_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128167 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128167 AND mptpi.`type`=4207 To do Qualite : 0.04314198433668906 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128168_01-02-2025_01_31_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128168 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128168 AND mptpi.`type`=4207 To do Qualite : 0.07726311033869263 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128169_01-02-2025_01_28_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128169 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20128169 AND mptpi.`type`=3726 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20128896_01-02-2025_06_58_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20128896 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'31012025': {'nb_upload': 28, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1333240473, 1333240469, 1333234645, 1333234627, 1333234617, 1333234610, 1333234606, 1333234598, 1333234551, 1333234548] Looping around the photos to save general results len do output : 1 /20176483Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240473', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333240469', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234645', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234627', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234617', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234610', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234606', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234598', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234551', None, None, None, None, None, '2539822') ('3459', None, None, None, None, None, None, None, '2539822') ('3459', None, '1333234548', None, None, None, None, None, '2539822') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014558076858520508 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.777470111846924 time spend to save output : 0.01477956771850586 total time spend for step 11 : 11.79224967956543 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 168.15user 111.27system 5:08.21elapsed 90%CPU (0avgtext+0avgdata 4359296maxresident)k 1702128inputs+183264outputs (4413major+11432959minor)pagefaults 0swaps