python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2899924 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 25>10 please execute split_portfolio.py -i 20785806 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2611194'] with mtr_portfolio_ids : ['20785806'] and first list_photo_ids : [] new path : /proc/2899924/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 2.8355889320373535 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Feb 25 01:18:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-25 01:18:35.058173: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-25 01:18:35.095268: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-25 01:18:35.098148: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f9fa4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-25 01:18:35.098237: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-25 01:18:35.105146: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-25 01:18:35.342633: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x39319340 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-25 01:18:35.342722: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-25 01:18:35.344805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-25 01:18:35.345310: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-25 01:18:35.350940: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-25 01:18:35.370100: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-25 01:18:35.370979: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-25 01:18:35.405021: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-25 01:18:35.411189: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-25 01:18:35.470097: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-25 01:18:35.472083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-25 01:18:35.472602: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-25 01:18:35.473618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-25 01:18:35.473639: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-25 01:18:35.473651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-25 01:18:35.475806: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9815 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-25 01:18:35.825460: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-25 01:18:35.825557: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-25 01:18:35.825579: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-25 01:18:35.825604: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-25 01:18:35.825622: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-25 01:18:35.825640: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-25 01:18:35.825657: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-25 01:18:35.825675: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-25 01:18:35.827269: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-25 01:18:35.828511: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-25 01:18:35.828547: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-25 01:18:35.828563: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-25 01:18:35.828577: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-25 01:18:35.828591: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-25 01:18:35.828605: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-25 01:18:35.828618: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-25 01:18:35.828632: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-25 01:18:35.829925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-25 01:18:35.829962: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-25 01:18:35.829971: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-25 01:18:35.829978: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-25 01:18:35.831327: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9815 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-25 01:18:47.317453: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-25 01:18:47.595678: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 34 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 Detection mask done ! Trying to reset tf kernel 2900314 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5304 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10593 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0009527206420898438 nb_pixel_total : 28096 time to create 1 rle with old method : 0.0336766242980957 length of segment : 174 time for calcul the mask position with numpy : 0.0007808208465576172 nb_pixel_total : 26104 time to create 1 rle with old method : 0.03105330467224121 length of segment : 505 time for calcul the mask position with numpy : 0.0004928112030029297 nb_pixel_total : 21360 time to create 1 rle with old method : 0.02459239959716797 length of segment : 327 time for calcul the mask position with numpy : 0.0015673637390136719 nb_pixel_total : 74267 time to create 1 rle with old method : 0.08510017395019531 length of segment : 322 time for calcul the mask position with numpy : 0.00021576881408691406 nb_pixel_total : 11123 time to create 1 rle with old method : 0.01398158073425293 length of segment : 105 time for calcul the mask position with numpy : 0.0013897418975830078 nb_pixel_total : 62744 time to create 1 rle with old method : 0.07217025756835938 length of segment : 314 time for calcul the mask position with numpy : 0.003870725631713867 nb_pixel_total : 110221 time to create 1 rle with old method : 0.1247701644897461 length of segment : 589 time for calcul the mask position with numpy : 0.0002639293670654297 nb_pixel_total : 12244 time to create 1 rle with old method : 0.014527320861816406 length of segment : 148 time for calcul the mask position with numpy : 0.0003859996795654297 nb_pixel_total : 18176 time to create 1 rle with old method : 0.02193450927734375 length of segment : 145 time for calcul the mask position with numpy : 0.00010180473327636719 nb_pixel_total : 2257 time to create 1 rle with old method : 0.0027806758880615234 length of segment : 73 time for calcul the mask position with numpy : 0.00014543533325195312 nb_pixel_total : 5589 time to create 1 rle with old method : 0.008143186569213867 length of segment : 70 time for calcul the mask position with numpy : 0.00013446807861328125 nb_pixel_total : 5098 time to create 1 rle with old method : 0.006224632263183594 length of segment : 67 time for calcul the mask position with numpy : 0.00047326087951660156 nb_pixel_total : 22840 time to create 1 rle with old method : 0.027802467346191406 length of segment : 205 time for calcul the mask position with numpy : 0.00013184547424316406 nb_pixel_total : 1199 time to create 1 rle with old method : 0.0018527507781982422 length of segment : 55 time for calcul the mask position with numpy : 0.000766754150390625 nb_pixel_total : 17530 time to create 1 rle with old method : 0.026989459991455078 length of segment : 203 time for calcul the mask position with numpy : 0.0001728534698486328 nb_pixel_total : 6110 time to create 1 rle with old method : 0.007406711578369141 length of segment : 70 time for calcul the mask position with numpy : 0.00047898292541503906 nb_pixel_total : 15206 time to create 1 rle with old method : 0.017744779586791992 length of segment : 205 time for calcul the mask position with numpy : 0.0001971721649169922 nb_pixel_total : 7427 time to create 1 rle with old method : 0.008998394012451172 length of segment : 120 time for calcul the mask position with numpy : 0.0002834796905517578 nb_pixel_total : 12646 time to create 1 rle with old method : 0.018598079681396484 length of segment : 135 time for calcul the mask position with numpy : 0.0013756752014160156 nb_pixel_total : 49663 time to create 1 rle with old method : 0.056337833404541016 length of segment : 316 time for calcul the mask position with numpy : 0.0005931854248046875 nb_pixel_total : 20024 time to create 1 rle with old method : 0.024760007858276367 length of segment : 264 time for calcul the mask position with numpy : 0.000568389892578125 nb_pixel_total : 17725 time to create 1 rle with old method : 0.02121758460998535 length of segment : 213 time for calcul the mask position with numpy : 0.0003230571746826172 nb_pixel_total : 10470 time to create 1 rle with old method : 0.014233112335205078 length of segment : 95 time for calcul the mask position with numpy : 0.00047588348388671875 nb_pixel_total : 16868 time to create 1 rle with old method : 0.022977590560913086 length of segment : 142 time for calcul the mask position with numpy : 0.0008130073547363281 nb_pixel_total : 20349 time to create 1 rle with old method : 0.02522897720336914 length of segment : 233 time for calcul the mask position with numpy : 0.0003330707550048828 nb_pixel_total : 11041 time to create 1 rle with old method : 0.016791582107543945 length of segment : 106 time for calcul the mask position with numpy : 0.00014281272888183594 nb_pixel_total : 3893 time to create 1 rle with old method : 0.00485539436340332 length of segment : 76 time for calcul the mask position with numpy : 0.0003001689910888672 nb_pixel_total : 7219 time to create 1 rle with old method : 0.010177373886108398 length of segment : 80 time for calcul the mask position with numpy : 0.0034151077270507812 nb_pixel_total : 133178 time to create 1 rle with old method : 0.15306377410888672 length of segment : 819 time for calcul the mask position with numpy : 0.0001201629638671875 nb_pixel_total : 1199 time to create 1 rle with old method : 0.0017783641815185547 length of segment : 55 time for calcul the mask position with numpy : 0.001295328140258789 nb_pixel_total : 44120 time to create 1 rle with old method : 0.055849313735961914 length of segment : 243 time for calcul the mask position with numpy : 0.003997802734375 nb_pixel_total : 154450 time to create 1 rle with new method : 0.015249967575073242 length of segment : 820 time for calcul the mask position with numpy : 0.0004839897155761719 nb_pixel_total : 15852 time to create 1 rle with old method : 0.018448591232299805 length of segment : 162 time for calcul the mask position with numpy : 0.0005791187286376953 nb_pixel_total : 25177 time to create 1 rle with old method : 0.029631853103637695 length of segment : 167 time for calcul the mask position with numpy : 0.00011324882507324219 nb_pixel_total : 1883 time to create 1 rle with old method : 0.0023984909057617188 length of segment : 47 time for calcul the mask position with numpy : 0.0004737377166748047 nb_pixel_total : 26819 time to create 1 rle with old method : 0.031226634979248047 length of segment : 193 time for calcul the mask position with numpy : 0.0011301040649414062 nb_pixel_total : 13954 time to create 1 rle with old method : 0.01693558692932129 length of segment : 189 time for calcul the mask position with numpy : 0.0021524429321289062 nb_pixel_total : 63096 time to create 1 rle with old method : 0.07435464859008789 length of segment : 322 time for calcul the mask position with numpy : 0.0009527206420898438 nb_pixel_total : 39960 time to create 1 rle with old method : 0.04729151725769043 length of segment : 316 time for calcul the mask position with numpy : 0.0018031597137451172 nb_pixel_total : 121410 time to create 1 rle with old method : 0.13615202903747559 length of segment : 369 time for calcul the mask position with numpy : 0.00044274330139160156 nb_pixel_total : 13330 time to create 1 rle with old method : 0.01613450050354004 length of segment : 172 time for calcul the mask position with numpy : 0.00045752525329589844 nb_pixel_total : 23050 time to create 1 rle with old method : 0.027401447296142578 length of segment : 168 time for calcul the mask position with numpy : 0.001954793930053711 nb_pixel_total : 128731 time to create 1 rle with old method : 0.15076947212219238 length of segment : 392 time for calcul the mask position with numpy : 0.00028252601623535156 nb_pixel_total : 9610 time to create 1 rle with old method : 0.012707233428955078 length of segment : 126 time for calcul the mask position with numpy : 0.0013079643249511719 nb_pixel_total : 67086 time to create 1 rle with old method : 0.10352230072021484 length of segment : 208 time for calcul the mask position with numpy : 0.0007121562957763672 nb_pixel_total : 28579 time to create 1 rle with old method : 0.03363537788391113 length of segment : 313 time for calcul the mask position with numpy : 0.00044274330139160156 nb_pixel_total : 22741 time to create 1 rle with old method : 0.02739405632019043 length of segment : 175 time for calcul the mask position with numpy : 0.0011944770812988281 nb_pixel_total : 80123 time to create 1 rle with old method : 0.09888219833374023 length of segment : 458 time for calcul the mask position with numpy : 0.0002684593200683594 nb_pixel_total : 13594 time to create 1 rle with old method : 0.016031265258789062 length of segment : 122 time for calcul the mask position with numpy : 0.00041794776916503906 nb_pixel_total : 17163 time to create 1 rle with old method : 0.020412445068359375 length of segment : 220 time for calcul the mask position with numpy : 0.0024535655975341797 nb_pixel_total : 138573 time to create 1 rle with old method : 0.15872526168823242 length of segment : 837 time for calcul the mask position with numpy : 0.00024890899658203125 nb_pixel_total : 5749 time to create 1 rle with old method : 0.007436275482177734 length of segment : 53 time for calcul the mask position with numpy : 0.0024492740631103516 nb_pixel_total : 109009 time to create 1 rle with old method : 0.1303253173828125 length of segment : 736 time for calcul the mask position with numpy : 0.0011794567108154297 nb_pixel_total : 60506 time to create 1 rle with old method : 0.0698080062866211 length of segment : 435 time for calcul the mask position with numpy : 0.0007402896881103516 nb_pixel_total : 26824 time to create 1 rle with old method : 0.03400087356567383 length of segment : 296 time for calcul the mask position with numpy : 0.0005903244018554688 nb_pixel_total : 33185 time to create 1 rle with old method : 0.03917431831359863 length of segment : 293 time for calcul the mask position with numpy : 0.00142669677734375 nb_pixel_total : 35571 time to create 1 rle with old method : 0.042180776596069336 length of segment : 322 time for calcul the mask position with numpy : 0.0006759166717529297 nb_pixel_total : 29937 time to create 1 rle with old method : 0.03551149368286133 length of segment : 189 time for calcul the mask position with numpy : 0.00014543533325195312 nb_pixel_total : 2996 time to create 1 rle with old method : 0.003633737564086914 length of segment : 116 time for calcul the mask position with numpy : 0.0020012855529785156 nb_pixel_total : 94728 time to create 1 rle with old method : 0.10811400413513184 length of segment : 469 time for calcul the mask position with numpy : 0.0015215873718261719 nb_pixel_total : 22976 time to create 1 rle with old method : 0.0269320011138916 length of segment : 439 time for calcul the mask position with numpy : 0.004338741302490234 nb_pixel_total : 139658 time to create 1 rle with old method : 0.161545991897583 length of segment : 472 time for calcul the mask position with numpy : 0.001123666763305664 nb_pixel_total : 32156 time to create 1 rle with old method : 0.0396270751953125 length of segment : 351 time for calcul the mask position with numpy : 0.001056671142578125 nb_pixel_total : 14347 time to create 1 rle with old method : 0.01692795753479004 length of segment : 164 time for calcul the mask position with numpy : 0.001729726791381836 nb_pixel_total : 22423 time to create 1 rle with old method : 0.026573657989501953 length of segment : 222 time for calcul the mask position with numpy : 0.00017452239990234375 nb_pixel_total : 1612 time to create 1 rle with old method : 0.0020751953125 length of segment : 35 time for calcul the mask position with numpy : 0.0015664100646972656 nb_pixel_total : 23739 time to create 1 rle with old method : 0.02742934226989746 length of segment : 250 time for calcul the mask position with numpy : 0.0076923370361328125 nb_pixel_total : 184204 time to create 1 rle with new method : 0.016717910766601562 length of segment : 766 time for calcul the mask position with numpy : 0.0030927658081054688 nb_pixel_total : 45179 time to create 1 rle with old method : 0.05347704887390137 length of segment : 464 time for calcul the mask position with numpy : 0.0002982616424560547 nb_pixel_total : 2410 time to create 1 rle with old method : 0.0030641555786132812 length of segment : 71 time for calcul the mask position with numpy : 0.003283977508544922 nb_pixel_total : 116097 time to create 1 rle with old method : 0.13298320770263672 length of segment : 400 time for calcul the mask position with numpy : 0.009491443634033203 nb_pixel_total : 297525 time to create 1 rle with new method : 0.022588253021240234 length of segment : 785 time for calcul the mask position with numpy : 0.0029230117797851562 nb_pixel_total : 48734 time to create 1 rle with old method : 0.0565187931060791 length of segment : 382 time for calcul the mask position with numpy : 0.0017685890197753906 nb_pixel_total : 55349 time to create 1 rle with old method : 0.0630791187286377 length of segment : 267 time for calcul the mask position with numpy : 0.006956338882446289 nb_pixel_total : 302569 time to create 1 rle with new method : 0.07690548896789551 length of segment : 908 time for calcul the mask position with numpy : 0.014226913452148438 nb_pixel_total : 356358 time to create 1 rle with new method : 0.07397174835205078 length of segment : 931 time for calcul the mask position with numpy : 0.00013327598571777344 nb_pixel_total : 2879 time to create 1 rle with old method : 0.0037162303924560547 length of segment : 44 time for calcul the mask position with numpy : 0.00029468536376953125 nb_pixel_total : 3632 time to create 1 rle with old method : 0.004494428634643555 length of segment : 240 time for calcul the mask position with numpy : 0.00492405891418457 nb_pixel_total : 271402 time to create 1 rle with new method : 0.07561469078063965 length of segment : 778 time for calcul the mask position with numpy : 0.00013518333435058594 nb_pixel_total : 3990 time to create 1 rle with old method : 0.004804372787475586 length of segment : 61 time for calcul the mask position with numpy : 0.00019025802612304688 nb_pixel_total : 3694 time to create 1 rle with old method : 0.004591703414916992 length of segment : 54 time for calcul the mask position with numpy : 0.000335693359375 nb_pixel_total : 13912 time to create 1 rle with old method : 0.016341686248779297 length of segment : 171 time for calcul the mask position with numpy : 0.0005295276641845703 nb_pixel_total : 20699 time to create 1 rle with old method : 0.024522066116333008 length of segment : 219 time for calcul the mask position with numpy : 0.0002753734588623047 nb_pixel_total : 8771 time to create 1 rle with old method : 0.010738134384155273 length of segment : 157 time for calcul the mask position with numpy : 0.0004200935363769531 nb_pixel_total : 22581 time to create 1 rle with old method : 0.02630901336669922 length of segment : 217 time for calcul the mask position with numpy : 0.0001518726348876953 nb_pixel_total : 5374 time to create 1 rle with old method : 0.006449222564697266 length of segment : 125 time for calcul the mask position with numpy : 0.0015978813171386719 nb_pixel_total : 104057 time to create 1 rle with old method : 0.11786365509033203 length of segment : 620 time for calcul the mask position with numpy : 0.0005114078521728516 nb_pixel_total : 26268 time to create 1 rle with old method : 0.030672788619995117 length of segment : 184 time for calcul the mask position with numpy : 0.0016832351684570312 nb_pixel_total : 132523 time to create 1 rle with old method : 0.15306782722473145 length of segment : 334 time for calcul the mask position with numpy : 0.0018382072448730469 nb_pixel_total : 125553 time to create 1 rle with old method : 0.14756155014038086 length of segment : 430 time for calcul the mask position with numpy : 0.000484466552734375 nb_pixel_total : 9119 time to create 1 rle with old method : 0.01053476333618164 length of segment : 188 time for calcul the mask position with numpy : 0.0021445751190185547 nb_pixel_total : 109209 time to create 1 rle with old method : 0.1509716510772705 length of segment : 439 time for calcul the mask position with numpy : 0.00049591064453125 nb_pixel_total : 7102 time to create 1 rle with old method : 0.008783102035522461 length of segment : 134 time for calcul the mask position with numpy : 0.0010538101196289062 nb_pixel_total : 31453 time to create 1 rle with old method : 0.03703880310058594 length of segment : 472 time for calcul the mask position with numpy : 0.004601716995239258 nb_pixel_total : 208960 time to create 1 rle with new method : 0.012285709381103516 length of segment : 617 time for calcul the mask position with numpy : 0.00022554397583007812 nb_pixel_total : 6644 time to create 1 rle with old method : 0.007872343063354492 length of segment : 144 time for calcul the mask position with numpy : 0.0006184577941894531 nb_pixel_total : 29618 time to create 1 rle with old method : 0.03495359420776367 length of segment : 261 time for calcul the mask position with numpy : 0.00012755393981933594 nb_pixel_total : 2372 time to create 1 rle with old method : 0.0029289722442626953 length of segment : 63 time for calcul the mask position with numpy : 0.00010943412780761719 nb_pixel_total : 2612 time to create 1 rle with old method : 0.0031499862670898438 length of segment : 123 time for calcul the mask position with numpy : 0.0007700920104980469 nb_pixel_total : 37526 time to create 1 rle with old method : 0.04337286949157715 length of segment : 386 time for calcul the mask position with numpy : 0.0009856224060058594 nb_pixel_total : 51604 time to create 1 rle with old method : 0.06145334243774414 length of segment : 647 time for calcul the mask position with numpy : 0.0005774497985839844 nb_pixel_total : 1984 time to create 1 rle with old method : 0.002782583236694336 length of segment : 122 time for calcul the mask position with numpy : 0.00017213821411132812 nb_pixel_total : 5598 time to create 1 rle with old method : 0.007178306579589844 length of segment : 80 time for calcul the mask position with numpy : 0.002486705780029297 nb_pixel_total : 80829 time to create 1 rle with old method : 0.09177994728088379 length of segment : 326 time for calcul the mask position with numpy : 0.0017817020416259766 nb_pixel_total : 28041 time to create 1 rle with old method : 0.03207135200500488 length of segment : 315 time for calcul the mask position with numpy : 0.0018074512481689453 nb_pixel_total : 51897 time to create 1 rle with old method : 0.05976414680480957 length of segment : 317 time for calcul the mask position with numpy : 0.00018310546875 nb_pixel_total : 2690 time to create 1 rle with old method : 0.0033164024353027344 length of segment : 59 time for calcul the mask position with numpy : 0.0005426406860351562 nb_pixel_total : 15092 time to create 1 rle with old method : 0.017775535583496094 length of segment : 140 time for calcul the mask position with numpy : 0.0013065338134765625 nb_pixel_total : 33063 time to create 1 rle with old method : 0.0386052131652832 length of segment : 318 time for calcul the mask position with numpy : 0.0018286705017089844 nb_pixel_total : 52960 time to create 1 rle with old method : 0.06036019325256348 length of segment : 349 time for calcul the mask position with numpy : 0.0001914501190185547 nb_pixel_total : 3296 time to create 1 rle with old method : 0.0041120052337646484 length of segment : 60 time for calcul the mask position with numpy : 0.0015897750854492188 nb_pixel_total : 45227 time to create 1 rle with old method : 0.054116249084472656 length of segment : 278 time for calcul the mask position with numpy : 0.0004973411560058594 nb_pixel_total : 9089 time to create 1 rle with old method : 0.010695219039916992 length of segment : 130 time for calcul the mask position with numpy : 0.003377676010131836 nb_pixel_total : 102361 time to create 1 rle with old method : 0.11514973640441895 length of segment : 920 time for calcul the mask position with numpy : 0.00017261505126953125 nb_pixel_total : 1062 time to create 1 rle with old method : 0.0014452934265136719 length of segment : 29 time for calcul the mask position with numpy : 0.000141143798828125 nb_pixel_total : 1020 time to create 1 rle with old method : 0.0013034343719482422 length of segment : 38 time for calcul the mask position with numpy : 0.0013642311096191406 nb_pixel_total : 27373 time to create 1 rle with old method : 0.03316187858581543 length of segment : 348 time for calcul the mask position with numpy : 0.0002391338348388672 nb_pixel_total : 10821 time to create 1 rle with old method : 0.012824773788452148 length of segment : 132 time for calcul the mask position with numpy : 0.002911806106567383 nb_pixel_total : 117085 time to create 1 rle with old method : 0.13407230377197266 length of segment : 417 time for calcul the mask position with numpy : 0.0006597042083740234 nb_pixel_total : 33273 time to create 1 rle with old method : 0.038630008697509766 length of segment : 170 time for calcul the mask position with numpy : 0.002565622329711914 nb_pixel_total : 166692 time to create 1 rle with new method : 0.0103302001953125 length of segment : 512 time for calcul the mask position with numpy : 0.0009093284606933594 nb_pixel_total : 53314 time to create 1 rle with old method : 0.061795711517333984 length of segment : 364 time for calcul the mask position with numpy : 0.0005693435668945312 nb_pixel_total : 19797 time to create 1 rle with old method : 0.023333072662353516 length of segment : 178 time for calcul the mask position with numpy : 0.0006642341613769531 nb_pixel_total : 45435 time to create 1 rle with old method : 0.05348062515258789 length of segment : 182 time for calcul the mask position with numpy : 0.0006833076477050781 nb_pixel_total : 38913 time to create 1 rle with old method : 0.04682612419128418 length of segment : 191 time for calcul the mask position with numpy : 0.0005896091461181641 nb_pixel_total : 41653 time to create 1 rle with old method : 0.05260515213012695 length of segment : 165 time for calcul the mask position with numpy : 0.002476930618286133 nb_pixel_total : 205292 time to create 1 rle with new method : 0.00580906867980957 length of segment : 484 time for calcul the mask position with numpy : 0.0002429485321044922 nb_pixel_total : 3329 time to create 1 rle with old method : 0.0042421817779541016 length of segment : 71 time for calcul the mask position with numpy : 0.0019228458404541016 nb_pixel_total : 58841 time to create 1 rle with old method : 0.06887125968933105 length of segment : 291 time for calcul the mask position with numpy : 0.0004296302795410156 nb_pixel_total : 20965 time to create 1 rle with old method : 0.02449941635131836 length of segment : 220 time for calcul the mask position with numpy : 0.00025272369384765625 nb_pixel_total : 3441 time to create 1 rle with old method : 0.004070281982421875 length of segment : 113 time for calcul the mask position with numpy : 0.00022530555725097656 nb_pixel_total : 4057 time to create 1 rle with old method : 0.0048177242279052734 length of segment : 76 time for calcul the mask position with numpy : 0.0002841949462890625 nb_pixel_total : 8081 time to create 1 rle with old method : 0.010074853897094727 length of segment : 76 time for calcul the mask position with numpy : 0.0009169578552246094 nb_pixel_total : 37730 time to create 1 rle with old method : 0.04404640197753906 length of segment : 254 time for calcul the mask position with numpy : 0.0002942085266113281 nb_pixel_total : 5996 time to create 1 rle with old method : 0.007268190383911133 length of segment : 93 time for calcul the mask position with numpy : 0.0020263195037841797 nb_pixel_total : 48841 time to create 1 rle with old method : 0.056130409240722656 length of segment : 374 time for calcul the mask position with numpy : 0.3669857978820801 nb_pixel_total : 1068600 time to create 1 rle with new method : 0.18140625953674316 length of segment : 1808 time for calcul the mask position with numpy : 0.002651214599609375 nb_pixel_total : 108054 time to create 1 rle with old method : 0.12035942077636719 length of segment : 479 time for calcul the mask position with numpy : 0.00018739700317382812 nb_pixel_total : 5152 time to create 1 rle with old method : 0.006979942321777344 length of segment : 35 time for calcul the mask position with numpy : 0.0047109127044677734 nb_pixel_total : 195781 time to create 1 rle with new method : 0.007251739501953125 length of segment : 661 time for calcul the mask position with numpy : 0.004195213317871094 nb_pixel_total : 200175 time to create 1 rle with new method : 0.006433248519897461 length of segment : 491 time for calcul the mask position with numpy : 0.0009124279022216797 nb_pixel_total : 38298 time to create 1 rle with old method : 0.06183147430419922 length of segment : 254 time for calcul the mask position with numpy : 0.0005385875701904297 nb_pixel_total : 9308 time to create 1 rle with old method : 0.011744976043701172 length of segment : 100 time for calcul the mask position with numpy : 0.00482630729675293 nb_pixel_total : 301965 time to create 1 rle with new method : 0.011128902435302734 length of segment : 449 time for calcul the mask position with numpy : 0.00019621849060058594 nb_pixel_total : 1597 time to create 1 rle with old method : 0.002071380615234375 length of segment : 47 time for calcul the mask position with numpy : 0.00016117095947265625 nb_pixel_total : 7788 time to create 1 rle with old method : 0.009169578552246094 length of segment : 144 time for calcul the mask position with numpy : 0.00031828880310058594 nb_pixel_total : 18070 time to create 1 rle with old method : 0.02119755744934082 length of segment : 160 time for calcul the mask position with numpy : 0.0002257823944091797 nb_pixel_total : 2925 time to create 1 rle with old method : 0.003868579864501953 length of segment : 64 time for calcul the mask position with numpy : 0.0003006458282470703 nb_pixel_total : 15890 time to create 1 rle with old method : 0.018507957458496094 length of segment : 149 time for calcul the mask position with numpy : 0.0015330314636230469 nb_pixel_total : 65372 time to create 1 rle with old method : 0.07528305053710938 length of segment : 334 time for calcul the mask position with numpy : 0.002366304397583008 nb_pixel_total : 104450 time to create 1 rle with old method : 0.14544415473937988 length of segment : 246 time for calcul the mask position with numpy : 0.000518798828125 nb_pixel_total : 34057 time to create 1 rle with old method : 0.03917956352233887 length of segment : 226 time for calcul the mask position with numpy : 0.0007596015930175781 nb_pixel_total : 27797 time to create 1 rle with old method : 0.03315448760986328 length of segment : 233 time spent for convertir_results : 13.45792841911316 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 158 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 43246 save missing photos in datou_result : time spend for datou_step_exec : 78.1079626083374 time spend to save output : 3.0883700847625732 total time spend for step 1 : 81.19633269309998 step2:crop_condition Tue Feb 25 01:19:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 158 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 42 About to insert : list_path_to_insert length 42 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339341887, 1339341886, 1339341880, 1339341874, 1339341854, 1339341853, 1339341852, 1339341851, 1339341849, 1339341847] Looping around the photos to save general results len do output : 92 /-3685660594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660633Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660635Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660627Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660613Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660665Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685879531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660596Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660622Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660624Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660631Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660636Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660638Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660658Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660672Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660651Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660660Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660653Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660656Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685879535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660657Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660629Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685660667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685673496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341887', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341886', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341880', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341874', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341854', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341853', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341852', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341851', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341849', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341847', None, None, None, None, None, '2611194') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 286 time used for this insertion : 0.03493642807006836 save_final save missing photos in datou_result : time spend for datou_step_exec : 30.07907772064209 time spend to save output : 0.055524587631225586 total time spend for step 2 : 30.134602308273315 step3:thcl Tue Feb 25 01:20:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.005720615386962891 time to convert the images to numpy array : 0.03670930862426758 time to import caffe and check if the image exist : 0.0072553157806396484 time to convert the images to numpy array : 0.05448436737060547 time to import caffe and check if the image exist : 0.0133819580078125 time to convert the images to numpy array : 0.0974123477935791 time to import caffe and check if the image exist : 0.005928754806518555 time to convert the images to numpy array : 0.10731172561645508 time to import caffe and check if the image exist : 0.011541366577148438 time to convert the images to numpy array : 0.12350058555603027 time to import caffe and check if the image exist : 0.010298490524291992 time to convert the images to numpy array : 0.12869834899902344 time to import caffe and check if the image exist : 0.009490251541137695 time to convert the images to numpy array : 0.13277983665466309 time to import caffe and check if the image exist : 0.013671636581420898 time to convert the images to numpy array : 0.1298670768737793 time to import caffe and check if the image exist : 0.012018442153930664 time to convert the images to numpy array : 0.13292312622070312 time to import caffe and check if the image exist : 0.012148618698120117 time to convert the images to numpy array : 0.14133095741271973 total time to convert the images to numpy array : 0.5287089347839355 list photo_ids error: [] list photo_ids correct : [-3685673487, -3685673496, -3685660603, -3685660617, -3685660616, -3685660637, -3685660657, -3685673461, -3685673484, -3685673482, -3685673483, -3685660606, -3685673458, -3685673457, -3685673472, -3685673475, -3685673478, -3685673479, -3685673485, -3685673500, -3685673499, -3685879535, -3685660594, -3685660633, -3685660635, -3685660647, -3685660654, -3685673455, -3685673460, -3685673470, -3685673481, -3685673494, -3685673463, -3685673462, -3685673497, -3685660601, -3685660625, -3685673446, -3685673466, -3685673473, -3685660599, -3685879531, -3685660608, -3685660632, -3685660627, -3685660613, -3685660630, -3685660643, -3685660665, -3685673437, -3685673451, -3685673452, -3685660596, -3685660622, -3685660624, -3685660631, -3685660615, -3685660614, -3685660640, -3685660636, -3685660638, -3685660646, -3685660597, -3685660629, -3685660667, -3685673439, -3685673441, -3685673476, -3685673474, -3685673471, -3685673501, -3685673491, -3685660658, -3685660662, -3685660672, -3685660651, -3685660660, -3685660664, -3685660655, -3685660653, -3685660656, -3685660648, -3685660666, -3685673443, -3685673436, -3685673438, -3685673444, -3685673450, -3685673449, -3685673453, -3685673459, -3685673454] number of photos to traite : 92 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10374 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0391771793365479 time used to do the prediction : 0.2537343502044678 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.9550538063049316 time used to do the prediction : 0.18636560440063477 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.8596649169921875e-05 save missing photos in datou_result : time spend for datou_step_exec : 10.64766240119934 time spend to save output : 0.0006573200225830078 total time spend for step 3 : 10.648319721221924 step4:merge_mask_thcl_custom Tue Feb 25 01:20:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 158 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1339341887, 1339341886, 1339341880, 1339341874, 1339341854, 1339341853, 1339341852, 1339341851, 1339341849, 1339341847] Looping around the photos to save general results len do output : 10 /1339341887Didn't retrieve data .Didn't retrieve data . /1339341886Didn't retrieve data .Didn't retrieve data . /1339341880Didn't retrieve data .Didn't retrieve data . /1339341874Didn't retrieve data .Didn't retrieve data . /1339341854Didn't retrieve data .Didn't retrieve data . /1339341853Didn't retrieve data .Didn't retrieve data . /1339341852Didn't retrieve data .Didn't retrieve data . /1339341851Didn't retrieve data .Didn't retrieve data . /1339341849Didn't retrieve data .Didn't retrieve data . /1339341847Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341887', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341886', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341880', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341874', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341854', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341853', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341852', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341851', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341849', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341847', None, None, None, None, None, '2611194') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.13344430923461914 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06522798538208008 time spend to save output : 0.13410449028015137 total time spend for step 4 : 0.19933247566223145 step5:rle_unique_nms_with_priority Tue Feb 25 01:20:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 17 nb_hashtags : 6 time to prepare the origin masks : 19.42032480239868 time for calcul the mask position with numpy : 0.3287346363067627 nb_pixel_total : 6619571 time to create 1 rle with new method : 0.3612995147705078 time for calcul the mask position with numpy : 0.04316568374633789 nb_pixel_total : 62651 time to create 1 rle with old method : 0.06962132453918457 time for calcul the mask position with numpy : 0.04508519172668457 nb_pixel_total : 12552 time to create 1 rle with old method : 0.013970375061035156 time for calcul the mask position with numpy : 0.044745445251464844 nb_pixel_total : 22916 time to create 1 rle with old method : 0.026057720184326172 time for calcul the mask position with numpy : 0.043329477310180664 nb_pixel_total : 110176 time to create 1 rle with old method : 0.12439680099487305 time for calcul the mask position with numpy : 0.041425228118896484 nb_pixel_total : 1189 time to create 1 rle with old method : 0.0013058185577392578 time for calcul the mask position with numpy : 0.0435028076171875 nb_pixel_total : 18119 time to create 1 rle with old method : 0.020375490188598633 time for calcul the mask position with numpy : 0.049630165100097656 nb_pixel_total : 219 time to create 1 rle with old method : 0.0006635189056396484 time for calcul the mask position with numpy : 0.04699850082397461 nb_pixel_total : 27982 time to create 1 rle with old method : 0.030498504638671875 time for calcul the mask position with numpy : 0.025617599487304688 nb_pixel_total : 17531 time to create 1 rle with old method : 0.02036142349243164 time for calcul the mask position with numpy : 0.025146007537841797 nb_pixel_total : 5586 time to create 1 rle with old method : 0.0064160823822021484 time for calcul the mask position with numpy : 0.026871919631958008 nb_pixel_total : 2246 time to create 1 rle with old method : 0.0025627613067626953 time for calcul the mask position with numpy : 0.02549600601196289 nb_pixel_total : 4881 time to create 1 rle with old method : 0.005387067794799805 time for calcul the mask position with numpy : 0.024994611740112305 nb_pixel_total : 21389 time to create 1 rle with old method : 0.022820711135864258 time for calcul the mask position with numpy : 0.027503013610839844 nb_pixel_total : 12198 time to create 1 rle with old method : 0.013492584228515625 time for calcul the mask position with numpy : 0.027361392974853516 nb_pixel_total : 11088 time to create 1 rle with old method : 0.01234579086303711 time for calcul the mask position with numpy : 0.024785995483398438 nb_pixel_total : 25954 time to create 1 rle with old method : 0.028983354568481445 time for calcul the mask position with numpy : 0.024077653884887695 nb_pixel_total : 73992 time to create 1 rle with old method : 0.08309388160705566 create new chi : 1.8001396656036377 time to delete rle : 0.021994590759277344 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 9132 TO DO : save crop sub photo not yet done ! save time : 0.5628235340118408 nb_obj : 20 nb_hashtags : 8 time to prepare the origin masks : 19.05507469177246 time for calcul the mask position with numpy : 0.860480546951294 nb_pixel_total : 6552494 time to create 1 rle with new method : 0.4645984172821045 time for calcul the mask position with numpy : 0.024280309677124023 nb_pixel_total : 17653 time to create 1 rle with old method : 0.02425694465637207 time for calcul the mask position with numpy : 0.02749466896057129 nb_pixel_total : 44039 time to create 1 rle with old method : 0.05969667434692383 time for calcul the mask position with numpy : 0.0279083251953125 nb_pixel_total : 38887 time to create 1 rle with old method : 0.046417236328125 time for calcul the mask position with numpy : 0.03419351577758789 nb_pixel_total : 543 time to create 1 rle with old method : 0.001177072525024414 time for calcul the mask position with numpy : 0.03012251853942871 nb_pixel_total : 16791 time to create 1 rle with old method : 0.019428014755249023 time for calcul the mask position with numpy : 0.02699136734008789 nb_pixel_total : 2258 time to create 1 rle with old method : 0.0028290748596191406 time for calcul the mask position with numpy : 0.03222918510437012 nb_pixel_total : 25100 time to create 1 rle with old method : 0.0315699577331543 time for calcul the mask position with numpy : 0.030871868133544922 nb_pixel_total : 154731 time to create 1 rle with new method : 0.5030484199523926 time for calcul the mask position with numpy : 0.023235559463500977 nb_pixel_total : 1878 time to create 1 rle with old method : 0.0021638870239257812 time for calcul the mask position with numpy : 0.03590226173400879 nb_pixel_total : 10420 time to create 1 rle with old method : 0.011877298355102539 time for calcul the mask position with numpy : 0.03734707832336426 nb_pixel_total : 15835 time to create 1 rle with old method : 0.019347429275512695 time for calcul the mask position with numpy : 0.035648345947265625 nb_pixel_total : 43050 time to create 1 rle with old method : 0.048749446868896484 time for calcul the mask position with numpy : 0.04174399375915527 nb_pixel_total : 19934 time to create 1 rle with old method : 0.023623228073120117 time for calcul the mask position with numpy : 0.035930633544921875 nb_pixel_total : 1198 time to create 1 rle with old method : 0.0013566017150878906 time for calcul the mask position with numpy : 0.036630868911743164 nb_pixel_total : 49334 time to create 1 rle with old method : 0.056977272033691406 time for calcul the mask position with numpy : 0.036156415939331055 nb_pixel_total : 7190 time to create 1 rle with old method : 0.00820302963256836 time for calcul the mask position with numpy : 0.03616619110107422 nb_pixel_total : 3885 time to create 1 rle with old method : 0.004560947418212891 time for calcul the mask position with numpy : 0.03440666198730469 nb_pixel_total : 10980 time to create 1 rle with old method : 0.01287388801574707 time for calcul the mask position with numpy : 0.03441119194030762 nb_pixel_total : 20170 time to create 1 rle with old method : 0.025193214416503906 time for calcul the mask position with numpy : 0.029737234115600586 nb_pixel_total : 13870 time to create 1 rle with old method : 0.015732526779174805 create new chi : 2.961538791656494 time to delete rle : 0.0011744499206542969 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 10232 TO DO : save crop sub photo not yet done ! save time : 0.5781769752502441 nb_obj : 12 nb_hashtags : 6 time to prepare the origin masks : 13.557319164276123 time for calcul the mask position with numpy : 1.2184510231018066 nb_pixel_total : 6503291 time to create 1 rle with new method : 0.4098074436187744 time for calcul the mask position with numpy : 0.02530980110168457 nb_pixel_total : 14046 time to create 1 rle with old method : 0.01534724235534668 time for calcul the mask position with numpy : 0.02470850944519043 nb_pixel_total : 9557 time to create 1 rle with old method : 0.010340690612792969 time for calcul the mask position with numpy : 0.027213573455810547 nb_pixel_total : 22624 time to create 1 rle with old method : 0.043166160583496094 time for calcul the mask position with numpy : 0.027727842330932617 nb_pixel_total : 5603 time to create 1 rle with old method : 0.0063021183013916016 time for calcul the mask position with numpy : 0.025365591049194336 nb_pixel_total : 28142 time to create 1 rle with old method : 0.030587196350097656 time for calcul the mask position with numpy : 0.02487492561340332 nb_pixel_total : 22935 time to create 1 rle with old method : 0.024331092834472656 time for calcul the mask position with numpy : 0.026328563690185547 nb_pixel_total : 121095 time to create 1 rle with old method : 0.1345839500427246 time for calcul the mask position with numpy : 0.02736353874206543 nb_pixel_total : 5377 time to create 1 rle with old method : 0.006232738494873047 time for calcul the mask position with numpy : 0.021821975708007812 nb_pixel_total : 66907 time to create 1 rle with old method : 0.07638072967529297 time for calcul the mask position with numpy : 0.02378559112548828 nb_pixel_total : 108902 time to create 1 rle with old method : 0.1404862403869629 time for calcul the mask position with numpy : 0.029099225997924805 nb_pixel_total : 128515 time to create 1 rle with old method : 0.15175914764404297 time for calcul the mask position with numpy : 0.027475833892822266 nb_pixel_total : 13246 time to create 1 rle with old method : 0.017581462860107422 create new chi : 2.6418075561523438 time to delete rle : 0.001346588134765625 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 8670 TO DO : save crop sub photo not yet done ! save time : 1.239973545074463 nb_obj : 28 nb_hashtags : 7 time to prepare the origin masks : 10.451568365097046 time for calcul the mask position with numpy : 0.6951384544372559 nb_pixel_total : 5402283 time to create 1 rle with new method : 0.5031702518463135 time for calcul the mask position with numpy : 0.045228004455566406 nb_pixel_total : 60412 time to create 1 rle with old method : 0.07184886932373047 time for calcul the mask position with numpy : 0.03379964828491211 nb_pixel_total : 183786 time to create 1 rle with new method : 0.7838730812072754 time for calcul the mask position with numpy : 0.02919769287109375 nb_pixel_total : 9920 time to create 1 rle with old method : 0.011541604995727539 time for calcul the mask position with numpy : 0.03007054328918457 nb_pixel_total : 37430 time to create 1 rle with old method : 0.04280209541320801 time for calcul the mask position with numpy : 0.03293609619140625 nb_pixel_total : 356218 time to create 1 rle with new method : 1.0159289836883545 time for calcul the mask position with numpy : 0.03121495246887207 nb_pixel_total : 270558 time to create 1 rle with new method : 0.6178994178771973 time for calcul the mask position with numpy : 0.030414342880249023 nb_pixel_total : 22278 time to create 1 rle with old method : 0.03287649154663086 time for calcul the mask position with numpy : 0.030093908309936523 nb_pixel_total : 17791 time to create 1 rle with old method : 0.020446062088012695 time for calcul the mask position with numpy : 0.030397415161132812 nb_pixel_total : 35248 time to create 1 rle with old method : 0.04024767875671387 time for calcul the mask position with numpy : 0.029537677764892578 nb_pixel_total : 1813 time to create 1 rle with old method : 0.0022139549255371094 time for calcul the mask position with numpy : 0.02889704704284668 nb_pixel_total : 48474 time to create 1 rle with old method : 0.0794377326965332 time for calcul the mask position with numpy : 0.03660106658935547 nb_pixel_total : 139357 time to create 1 rle with old method : 0.16526222229003906 time for calcul the mask position with numpy : 0.029187440872192383 nb_pixel_total : 32054 time to create 1 rle with old method : 0.03775644302368164 time for calcul the mask position with numpy : 0.029026508331298828 nb_pixel_total : 55258 time to create 1 rle with old method : 0.06431698799133301 time for calcul the mask position with numpy : 0.03412175178527832 nb_pixel_total : 116053 time to create 1 rle with old method : 0.14658808708190918 time for calcul the mask position with numpy : 0.03150773048400879 nb_pixel_total : 23616 time to create 1 rle with old method : 0.027113676071166992 time for calcul the mask position with numpy : 0.029352903366088867 nb_pixel_total : 1604 time to create 1 rle with old method : 0.0018477439880371094 time for calcul the mask position with numpy : 0.031549692153930664 nb_pixel_total : 3585 time to create 1 rle with old method : 0.004407644271850586 time for calcul the mask position with numpy : 0.02980828285217285 nb_pixel_total : 12280 time to create 1 rle with old method : 0.013908624649047852 time for calcul the mask position with numpy : 0.029453754425048828 nb_pixel_total : 2396 time to create 1 rle with old method : 0.0027294158935546875 time for calcul the mask position with numpy : 0.029348134994506836 nb_pixel_total : 3672 time to create 1 rle with old method : 0.0042116641998291016 time for calcul the mask position with numpy : 0.029489994049072266 nb_pixel_total : 29837 time to create 1 rle with old method : 0.03832840919494629 time for calcul the mask position with numpy : 0.0325007438659668 nb_pixel_total : 23044 time to create 1 rle with old method : 0.026012659072875977 time for calcul the mask position with numpy : 0.028757095336914062 nb_pixel_total : 26697 time to create 1 rle with old method : 0.03020024299621582 time for calcul the mask position with numpy : 0.031804561614990234 nb_pixel_total : 94543 time to create 1 rle with old method : 0.11501240730285645 time for calcul the mask position with numpy : 0.029104232788085938 nb_pixel_total : 33089 time to create 1 rle with old method : 0.038640737533569336 time for calcul the mask position with numpy : 0.029111862182617188 nb_pixel_total : 2983 time to create 1 rle with old method : 0.0035371780395507812 time for calcul the mask position with numpy : 0.03211212158203125 nb_pixel_total : 3961 time to create 1 rle with old method : 0.004483699798583984 create new chi : 5.639943599700928 time to delete rle : 0.001903533935546875 batch 1 Loaded 29 chid ids of type : 3726 Number RLEs to save : 20399 TO DO : save crop sub photo not yet done ! save time : 1.990006923675537 nb_obj : 13 nb_hashtags : 5 time to prepare the origin masks : 9.878066539764404 time for calcul the mask position with numpy : 0.9875190258026123 nb_pixel_total : 6257163 time to create 1 rle with new method : 0.3483750820159912 time for calcul the mask position with numpy : 0.023230314254760742 nb_pixel_total : 130855 time to create 1 rle with old method : 0.1517939567565918 time for calcul the mask position with numpy : 0.02588653564453125 nb_pixel_total : 110993 time to create 1 rle with old method : 0.12703418731689453 time for calcul the mask position with numpy : 0.022359371185302734 nb_pixel_total : 7088 time to create 1 rle with old method : 0.007567882537841797 time for calcul the mask position with numpy : 0.02645397186279297 nb_pixel_total : 125147 time to create 1 rle with old method : 0.1388545036315918 time for calcul the mask position with numpy : 0.025202512741088867 nb_pixel_total : 24148 time to create 1 rle with old method : 0.027137041091918945 time for calcul the mask position with numpy : 0.0257875919342041 nb_pixel_total : 208639 time to create 1 rle with new method : 0.8783347606658936 time for calcul the mask position with numpy : 0.02325153350830078 nb_pixel_total : 9039 time to create 1 rle with old method : 0.009618282318115234 time for calcul the mask position with numpy : 0.024403095245361328 nb_pixel_total : 5364 time to create 1 rle with old method : 0.0069353580474853516 time for calcul the mask position with numpy : 0.0259554386138916 nb_pixel_total : 31045 time to create 1 rle with old method : 0.03451371192932129 time for calcul the mask position with numpy : 0.021218299865722656 nb_pixel_total : 526 time to create 1 rle with old method : 0.0007216930389404297 time for calcul the mask position with numpy : 0.020800352096557617 nb_pixel_total : 22531 time to create 1 rle with old method : 0.02519512176513672 time for calcul the mask position with numpy : 0.022492647171020508 nb_pixel_total : 108946 time to create 1 rle with old method : 0.11849427223205566 time for calcul the mask position with numpy : 0.021843910217285156 nb_pixel_total : 8756 time to create 1 rle with old method : 0.009485483169555664 create new chi : 3.2410531044006348 time to delete rle : 0.0009658336639404297 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 9753 TO DO : save crop sub photo not yet done ! save time : 1.0059688091278076 nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 6.691187620162964 time for calcul the mask position with numpy : 0.9450170993804932 nb_pixel_total : 6913753 time to create 1 rle with new method : 0.6288096904754639 time for calcul the mask position with numpy : 0.03419351577758789 nb_pixel_total : 37420 time to create 1 rle with old method : 0.04190874099731445 time for calcul the mask position with numpy : 0.035298824310302734 nb_pixel_total : 29536 time to create 1 rle with old method : 0.033815622329711914 time for calcul the mask position with numpy : 0.03560304641723633 nb_pixel_total : 5569 time to create 1 rle with old method : 0.006682872772216797 time for calcul the mask position with numpy : 0.02213764190673828 nb_pixel_total : 2604 time to create 1 rle with old method : 0.0030350685119628906 time for calcul the mask position with numpy : 0.0233306884765625 nb_pixel_total : 50410 time to create 1 rle with old method : 0.0565640926361084 time for calcul the mask position with numpy : 0.022723674774169922 nb_pixel_total : 2353 time to create 1 rle with old method : 0.003159761428833008 time for calcul the mask position with numpy : 0.02206134796142578 nb_pixel_total : 1967 time to create 1 rle with old method : 0.0023131370544433594 time for calcul the mask position with numpy : 0.021000146865844727 nb_pixel_total : 6628 time to create 1 rle with old method : 0.007474184036254883 create new chi : 1.9854521751403809 time to delete rle : 0.0006153583526611328 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 5746 TO DO : save crop sub photo not yet done ! save time : 0.5287420749664307 nb_obj : 15 nb_hashtags : 6 time to prepare the origin masks : 10.984500408172607 time for calcul the mask position with numpy : 0.5414943695068359 nb_pixel_total : 6489071 time to create 1 rle with new method : 0.562225341796875 time for calcul the mask position with numpy : 0.022332429885864258 nb_pixel_total : 15054 time to create 1 rle with old method : 0.016891002655029297 time for calcul the mask position with numpy : 0.040027618408203125 nb_pixel_total : 27934 time to create 1 rle with old method : 0.03467583656311035 time for calcul the mask position with numpy : 0.03850436210632324 nb_pixel_total : 116610 time to create 1 rle with old method : 0.13240861892700195 time for calcul the mask position with numpy : 0.03468060493469238 nb_pixel_total : 1013 time to create 1 rle with old method : 0.0010950565338134766 time for calcul the mask position with numpy : 0.03394436836242676 nb_pixel_total : 1046 time to create 1 rle with old method : 0.0012402534484863281 time for calcul the mask position with numpy : 0.03562736511230469 nb_pixel_total : 101749 time to create 1 rle with old method : 0.12076520919799805 time for calcul the mask position with numpy : 0.040482521057128906 nb_pixel_total : 33023 time to create 1 rle with old method : 0.039107322692871094 time for calcul the mask position with numpy : 0.03832864761352539 nb_pixel_total : 3294 time to create 1 rle with old method : 0.003761768341064453 time for calcul the mask position with numpy : 0.03803205490112305 nb_pixel_total : 1521 time to create 1 rle with old method : 0.001874685287475586 time for calcul the mask position with numpy : 0.04010796546936035 nb_pixel_total : 27022 time to create 1 rle with old method : 0.03271007537841797 time for calcul the mask position with numpy : 0.03934788703918457 nb_pixel_total : 2668 time to create 1 rle with old method : 0.0035490989685058594 time for calcul the mask position with numpy : 0.03921318054199219 nb_pixel_total : 45042 time to create 1 rle with old method : 0.06327986717224121 time for calcul the mask position with numpy : 0.03084731101989746 nb_pixel_total : 80544 time to create 1 rle with old method : 0.1021263599395752 time for calcul the mask position with numpy : 0.04073166847229004 nb_pixel_total : 52829 time to create 1 rle with old method : 0.06920027732849121 time for calcul the mask position with numpy : 0.022458314895629883 nb_pixel_total : 51820 time to create 1 rle with old method : 0.06276130676269531 create new chi : 2.3652753829956055 time to delete rle : 0.001493692398071289 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 10098 TO DO : save crop sub photo not yet done ! save time : 1.2682623863220215 nb_obj : 8 nb_hashtags : 5 time to prepare the origin masks : 7.358582258224487 time for calcul the mask position with numpy : 0.3066294193267822 nb_pixel_total : 6489039 time to create 1 rle with new method : 0.38338422775268555 time for calcul the mask position with numpy : 0.04669332504272461 nb_pixel_total : 204862 time to create 1 rle with new method : 0.49074554443359375 time for calcul the mask position with numpy : 0.04237771034240723 nb_pixel_total : 165871 time to create 1 rle with new method : 0.38921022415161133 time for calcul the mask position with numpy : 0.0242464542388916 nb_pixel_total : 3839 time to create 1 rle with old method : 0.004338264465332031 time for calcul the mask position with numpy : 0.024764299392700195 nb_pixel_total : 41619 time to create 1 rle with old method : 0.04430127143859863 time for calcul the mask position with numpy : 0.024021387100219727 nb_pixel_total : 53311 time to create 1 rle with old method : 0.05762147903442383 time for calcul the mask position with numpy : 0.025073766708374023 nb_pixel_total : 38862 time to create 1 rle with old method : 0.041613101959228516 time for calcul the mask position with numpy : 0.02530503273010254 nb_pixel_total : 19730 time to create 1 rle with old method : 0.020821571350097656 time for calcul the mask position with numpy : 0.02401256561279297 nb_pixel_total : 33107 time to create 1 rle with old method : 0.03547048568725586 create new chi : 2.103518009185791 time to delete rle : 0.0008242130279541016 batch 1 Loaded 11 chid ids of type : 3726 Number RLEs to save : 6561 TO DO : save crop sub photo not yet done ! save time : 0.510108470916748 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 5.439800024032593 time for calcul the mask position with numpy : 2.383037805557251 nb_pixel_total : 6908226 time to create 1 rle with new method : 1.2850806713104248 time for calcul the mask position with numpy : 0.032842159271240234 nb_pixel_total : 5941 time to create 1 rle with old method : 0.006667137145996094 time for calcul the mask position with numpy : 0.03568720817565918 nb_pixel_total : 58589 time to create 1 rle with old method : 0.061205148696899414 time for calcul the mask position with numpy : 0.033453941345214844 nb_pixel_total : 8022 time to create 1 rle with old method : 0.009687185287475586 time for calcul the mask position with numpy : 0.024973392486572266 nb_pixel_total : 4043 time to create 1 rle with old method : 0.004219770431518555 time for calcul the mask position with numpy : 0.025826215744018555 nb_pixel_total : 3434 time to create 1 rle with old method : 0.0037207603454589844 time for calcul the mask position with numpy : 0.02417778968811035 nb_pixel_total : 37687 time to create 1 rle with old method : 0.0390172004699707 time for calcul the mask position with numpy : 0.025182485580444336 nb_pixel_total : 88 time to create 1 rle with old method : 0.0001938343048095703 time for calcul the mask position with numpy : 0.02501702308654785 nb_pixel_total : 3259 time to create 1 rle with old method : 0.0034809112548828125 time for calcul the mask position with numpy : 0.0255739688873291 nb_pixel_total : 20951 time to create 1 rle with old method : 0.026751041412353516 create new chi : 4.139712572097778 time to delete rle : 0.023458242416381836 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 4594 TO DO : save crop sub photo not yet done ! save time : 0.800936222076416 nb_obj : 17 nb_hashtags : 6 time to prepare the origin masks : 17.20033311843872 time for calcul the mask position with numpy : 0.6993494033813477 nb_pixel_total : 5127352 time to create 1 rle with new method : 1.1476593017578125 time for calcul the mask position with numpy : 0.03724050521850586 nb_pixel_total : 48482 time to create 1 rle with old method : 0.06067061424255371 time for calcul the mask position with numpy : 0.03600311279296875 nb_pixel_total : 2893 time to create 1 rle with old method : 0.003437519073486328 time for calcul the mask position with numpy : 0.04222464561462402 nb_pixel_total : 1347 time to create 1 rle with old method : 0.0016524791717529297 time for calcul the mask position with numpy : 0.047046661376953125 nb_pixel_total : 27769 time to create 1 rle with old method : 0.0313260555267334 time for calcul the mask position with numpy : 0.03725767135620117 nb_pixel_total : 1597 time to create 1 rle with old method : 0.0018100738525390625 time for calcul the mask position with numpy : 0.037598609924316406 nb_pixel_total : 7765 time to create 1 rle with old method : 0.009001731872558594 time for calcul the mask position with numpy : 0.036629676818847656 nb_pixel_total : 481 time to create 1 rle with old method : 0.0007464885711669922 time for calcul the mask position with numpy : 0.03634810447692871 nb_pixel_total : 51494 time to create 1 rle with old method : 0.06033587455749512 time for calcul the mask position with numpy : 0.04467058181762695 nb_pixel_total : 9266 time to create 1 rle with old method : 0.011438608169555664 time for calcul the mask position with numpy : 0.047532081604003906 nb_pixel_total : 199926 time to create 1 rle with new method : 0.8052358627319336 time for calcul the mask position with numpy : 0.03281664848327637 nb_pixel_total : 195561 time to create 1 rle with new method : 0.6093864440917969 time for calcul the mask position with numpy : 0.029677152633666992 nb_pixel_total : 2110 time to create 1 rle with old method : 0.002422809600830078 time for calcul the mask position with numpy : 0.03673505783081055 nb_pixel_total : 1062877 time to create 1 rle with new method : 0.5692203044891357 time for calcul the mask position with numpy : 0.02285480499267578 nb_pixel_total : 107943 time to create 1 rle with old method : 0.12438726425170898 time for calcul the mask position with numpy : 0.023296594619750977 nb_pixel_total : 65235 time to create 1 rle with old method : 0.07099390029907227 time for calcul the mask position with numpy : 0.021775484085083008 nb_pixel_total : 104135 time to create 1 rle with old method : 0.11280655860900879 time for calcul the mask position with numpy : 0.02142930030822754 nb_pixel_total : 34007 time to create 1 rle with old method : 0.038635969161987305 create new chi : 5.06007981300354 time to delete rle : 0.0012619495391845703 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 13033 TO DO : save crop sub photo not yet done ! save time : 1.5244114398956299 map_output_result : {1339341887: (0.0, 'Should be the crop_list due to order', 0), 1339341886: (0.0, 'Should be the crop_list due to order', 0), 1339341880: (0.0, 'Should be the crop_list due to order', 0), 1339341874: (0.0, 'Should be the crop_list due to order', 0), 1339341854: (0.0, 'Should be the crop_list due to order', 0), 1339341853: (0.0, 'Should be the crop_list due to order', 0), 1339341852: (0.0, 'Should be the crop_list due to order', 0), 1339341851: (0.0, 'Should be the crop_list due to order', 0), 1339341849: (0.0, 'Should be the crop_list due to order', 0), 1339341847: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1339341887, 1339341886, 1339341880, 1339341874, 1339341854, 1339341853, 1339341852, 1339341851, 1339341849, 1339341847] Looping around the photos to save general results len do output : 10 /1339341887.Didn't retrieve data . /1339341886.Didn't retrieve data . /1339341880.Didn't retrieve data . /1339341874.Didn't retrieve data . /1339341854.Didn't retrieve data . /1339341853.Didn't retrieve data . /1339341852.Didn't retrieve data . /1339341851.Didn't retrieve data . /1339341849.Didn't retrieve data . /1339341847.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341887', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341886', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341880', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341874', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341854', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341853', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341852', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341851', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341849', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341847', None, None, None, None, None, '2611194') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.2950735092163086 save_final save missing photos in datou_result : time spend for datou_step_exec : 164.3494143486023 time spend to save output : 0.2998523712158203 total time spend for step 5 : 164.64926671981812 step6:crop_condition Tue Feb 25 01:23:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 164 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740443000_2899924 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.4531562328338623 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740443005_2899924 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.2130022048950195 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740443011_2899924 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.421501636505127 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740443016_2899924 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.9767117500305176 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740443020_2899924 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.5103788375854492 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740443024_2899924 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8552491664886475 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339341887, 1339341886, 1339341880, 1339341874, 1339341854, 1339341853, 1339341852, 1339341851, 1339341849, 1339341847] Looping around the photos to save general results len do output : 33 /1339459501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459524Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459535Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459536Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459537Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459541Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459569Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459581Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339459585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341887', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341886', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341880', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341874', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341854', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341853', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341852', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341851', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341849', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341847', None, None, None, None, None, '2611194') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 109 time used for this insertion : 0.6204633712768555 save_final save missing photos in datou_result : time spend for datou_step_exec : 28.703792095184326 time spend to save output : 0.6217920780181885 total time spend for step 6 : 29.325584173202515 step7:ventilate_hashtags_in_portfolio Tue Feb 25 01:23:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20785806 get user id for portfolio 20785806 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785806 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','mal_croppe','kraft','plastique','Teint_Dans_La_Masse','papier','flou','Carton_gris','autre_refus','environnement','metal','Carton_brun')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785806 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','mal_croppe','kraft','plastique','Teint_Dans_La_Masse','papier','flou','Carton_gris','autre_refus','environnement','metal','Carton_brun')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20785915,20785916,20785917,20785918,20785919,20785920,20785921,20785922,20785923,20785924,20785925,20785926?tags=Teint_Dans_La_Masse,environnement,metal,plastique,cartonnette,Carton_brun,papier,kraft,flou,Carton_gris,autre_refus,mal_croppe Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1339341887, 1339341886, 1339341880, 1339341874, 1339341854, 1339341853, 1339341852, 1339341851, 1339341849, 1339341847] Looping around the photos to save general results len do output : 1 /20785806. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341887', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341886', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341880', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341874', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341854', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341853', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341852', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341851', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341849', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341847', None, None, None, None, None, '2611194') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.1785285472869873 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.3805265426635742 time spend to save output : 0.17882704734802246 total time spend for step 7 : 1.5593535900115967 step8:final Tue Feb 25 01:23:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1339341887: ('0.030869787498143886',), 1339341886: ('0.030869787498143886',), 1339341880: ('0.030869787498143886',), 1339341874: ('0.030869787498143886',), 1339341854: ('0.030869787498143886',), 1339341853: ('0.030869787498143886',), 1339341852: ('0.030869787498143886',), 1339341851: ('0.030869787498143886',), 1339341849: ('0.030869787498143886',), 1339341847: ('0.030869787498143886',)} new output for save of step final : {1339341887: ('0.030869787498143886',), 1339341886: ('0.030869787498143886',), 1339341880: ('0.030869787498143886',), 1339341874: ('0.030869787498143886',), 1339341854: ('0.030869787498143886',), 1339341853: ('0.030869787498143886',), 1339341852: ('0.030869787498143886',), 1339341851: ('0.030869787498143886',), 1339341849: ('0.030869787498143886',), 1339341847: ('0.030869787498143886',)} [1339341887, 1339341886, 1339341880, 1339341874, 1339341854, 1339341853, 1339341852, 1339341851, 1339341849, 1339341847] Looping around the photos to save general results len do output : 10 /1339341887.Didn't retrieve data . /1339341886.Didn't retrieve data . /1339341880.Didn't retrieve data . /1339341874.Didn't retrieve data . /1339341854.Didn't retrieve data . /1339341853.Didn't retrieve data . /1339341852.Didn't retrieve data . /1339341851.Didn't retrieve data . /1339341849.Didn't retrieve data . /1339341847.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341887', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341886', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341880', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341874', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341854', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341853', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341852', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341851', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341849', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341847', None, None, None, None, None, '2611194') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013866662979125977 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3889155387878418 time spend to save output : 0.015144109725952148 total time spend for step 8 : 0.40405964851379395 step9:velours_tree Tue Feb 25 01:23:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.22092652320861816 time spend to save output : 4.9591064453125e-05 total time spend for step 9 : 0.2209761142730713 step10:send_mail_cod Tue Feb 25 01:23:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20785806_25-02-2025_01_23_49.pdf 20785917 imagette207859171740443029 20785918 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207859181740443029 20785919 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207859191740443031 20785922 change filename to text .imagette207859221740443032 20785923 imagette207859231740443032 20785925 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207859251740443032 20785926 imagette207859261740443033 20785915 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207859151740443033 20785920 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207859201740443034 20785924 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207859241740443035 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20785806 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20785915,20785916,20785917,20785918,20785919,20785920,20785921,20785922,20785923,20785924,20785925,20785926?tags=Teint_Dans_La_Masse,environnement,metal,plastique,cartonnette,Carton_brun,papier,kraft,flou,Carton_gris,autre_refus,mal_croppe your option no_mail is active, we will not send the real mail to your client args[1339341887] : ('0.030869787498143886',) no score found for photo 1339341887 We are sending mail with results at report@fotonower.com args[1339341886] : ('0.030869787498143886',) no score found for photo 1339341886 We are sending mail with results at report@fotonower.com args[1339341880] : ('0.030869787498143886',) no score found for photo 1339341880 We are sending mail with results at report@fotonower.com args[1339341874] : ('0.030869787498143886',) no score found for photo 1339341874 We are sending mail with results at report@fotonower.com args[1339341854] : ('0.030869787498143886',) no score found for photo 1339341854 We are sending mail with results at report@fotonower.com args[1339341853] : ('0.030869787498143886',) no score found for photo 1339341853 We are sending mail with results at report@fotonower.com args[1339341852] : ('0.030869787498143886',) no score found for photo 1339341852 We are sending mail with results at report@fotonower.com args[1339341851] : ('0.030869787498143886',) no score found for photo 1339341851 We are sending mail with results at report@fotonower.com args[1339341849] : ('0.030869787498143886',) no score found for photo 1339341849 We are sending mail with results at report@fotonower.com args[1339341847] : ('0.030869787498143886',) no score found for photo 1339341847 We are sending mail with results at report@fotonower.com refus_total : 0.030869787498143886 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20785806 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1339341825,1339346898,1339346895,1339346852,1339341887,1339341886,1339341880,1339341874,1339341854,1339341853,1339341852,1339341851,1339341847,1339341827,1339341826,1339341849,1339341829,1339346850,1339346851,1339346853) Found this number of photos: 20 begin to download photo : 1339341825 begin to download photo : 1339341886 begin to download photo : 1339341852 begin to download photo : 1339341849 download finish for photo 1339341852 begin to download photo : 1339341851 download finish for photo 1339341849 begin to download photo : 1339341829 download finish for photo 1339341825 begin to download photo : 1339346898 download finish for photo 1339341886 begin to download photo : 1339341880 download finish for photo 1339341829 begin to download photo : 1339346850 download finish for photo 1339341851 begin to download photo : 1339341847 download finish for photo 1339341880 begin to download photo : 1339341874 download finish for photo 1339346850 begin to download photo : 1339346851 download finish for photo 1339341847 begin to download photo : 1339341827 download finish for photo 1339346898 begin to download photo : 1339346895 download finish for photo 1339341874 begin to download photo : 1339341854 download finish for photo 1339346851 begin to download photo : 1339346853 download finish for photo 1339341827 begin to download photo : 1339341826 download finish for photo 1339346895 begin to download photo : 1339346852 download finish for photo 1339346853 download finish for photo 1339341854 begin to download photo : 1339341853 download finish for photo 1339341826 download finish for photo 1339346852 begin to download photo : 1339341887 download finish for photo 1339341853 download finish for photo 1339341887 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785806_25-02-2025_01_23_49.pdf results_Auto_P20785806_25-02-2025_01_23_49.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785806_25-02-2025_01_23_49.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20785806','results_Auto_P20785806_25-02-2025_01_23_49.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785806_25-02-2025_01_23_49.pdf','pdf','','0.89','0.030869787498143886') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1339341887, 1339341886, 1339341880, 1339341874, 1339341854, 1339341853, 1339341852, 1339341851, 1339341849, 1339341847] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341887', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341886', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341880', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341874', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341854', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341853', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341852', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341851', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341849', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341847', None, None, None, None, None, '2611194') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.024777889251708984 save_final save missing photos in datou_result : time spend for datou_step_exec : 11.495189905166626 time spend to save output : 0.024970054626464844 total time spend for step 10 : 11.52015995979309 step11:split_time_score Tue Feb 25 01:24:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('13', 25),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 24022025 20785806 Nombre de photos uploadées : 25 / 23040 (0%) 24022025 20785806 Nombre de photos taguées (types de déchets): 0 / 25 (0%) 24022025 20785806 Nombre de photos taguées (volume) : 0 / 25 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 5.9604644775390625e-06 ????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0016727447509765625 elapsed_time : insert_dashboard_record_day_entry 0.03427410125732422 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785553 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785554 order by id desc limit 1 Qualite : 0.1371405894618548 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785555_25-02-2025_01_18_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785555 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785555 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785556 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785557 order by id desc limit 1 Qualite : 0.06979736276336668 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785558_25-02-2025_00_59_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785558 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785558 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785559 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785560 order by id desc limit 1 Qualite : 0.48062243696575063 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785561_25-02-2025_01_14_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785561 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785561 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785562 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785563 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785564 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785803 order by id desc limit 1 Qualite : 0.05972239396043188 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785566_25-02-2025_01_19_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785566 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785566 AND mptpi.`type`=4211 To do Qualite : 0.39292824224678974 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785567_25-02-2025_01_02_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785567 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785567 AND mptpi.`type`=4200 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782984 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785568 order by id desc limit 1 Qualite : 0.22267187071827646 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20782986_24-02-2025_22_36_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782986 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782986 AND mptpi.`type`=4200 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782987 order by id desc limit 1 Qualite : 0.12499677790385096 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20782988_24-02-2025_22_23_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782988 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782988 AND mptpi.`type`=4200 To do Qualite : 0.2106218921467765 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782989 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782989 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782990 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785804 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785805 order by id desc limit 1 Qualite : 0.031115119818097666 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785806_25-02-2025_01_23_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785806 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785806 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782994 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782995 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782996 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785807 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785808 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785809 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785810 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785811 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785812 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785569 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785570 order by id desc limit 1 Qualite : 0.1981934826399696 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785571 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785571 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785572 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785573 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785574 order by id desc limit 1 Qualite : 0.048232044229356064 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785829_25-02-2025_01_21_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785829 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785829 AND mptpi.`type`=4207 To do Qualite : 0.08155285407512981 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785576_25-02-2025_01_17_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785576 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785576 AND mptpi.`type`=4207 To do Qualite : 0.06481021225928252 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785577_25-02-2025_01_08_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785577 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785577 AND mptpi.`type`=4207 To do Qualite : 0.21349496612881258 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785578_25-02-2025_01_18_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785578 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785578 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785579_25-02-2025_01_12_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785579 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783011 order by id desc limit 1 Qualite : 0.20103672889149876 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20783012_24-02-2025_23_45_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783012 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20783012 AND mptpi.`type`=3594 To do Qualite : 0.052401843008330756 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20783013_24-02-2025_22_36_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783013 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20783013 AND mptpi.`type`=3726 To do Qualite : 0.10145267883062205 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20783014_24-02-2025_23_25_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783014 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20783014 AND mptpi.`type`=4207 To do Qualite : 0.056568829757248966 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20783015_24-02-2025_22_49_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783015 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20783015 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'24022025': {'nb_upload': 25, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1339341887, 1339341886, 1339341880, 1339341874, 1339341854, 1339341853, 1339341852, 1339341851, 1339341849, 1339341847] Looping around the photos to save general results len do output : 1 /20785806Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341887', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341886', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341880', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341874', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341854', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341853', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341852', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341851', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341849', None, None, None, None, None, '2611194') ('3459', None, None, None, None, None, None, None, '2611194') ('3459', None, '1339341847', None, None, None, None, None, '2611194') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.06774020195007324 save_final save missing photos in datou_result : time spend for datou_step_exec : 27.53580069541931 time spend to save output : 0.0679931640625 total time spend for step 11 : 27.60379385948181 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 137.72user 172.48system 6:04.91elapsed 85%CPU (0avgtext+0avgdata 4305368maxresident)k 2805888inputs+136336outputs (28372major+13143259minor)pagefaults 0swaps