python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 930758 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 17>10 please execute split_portfolio.py -i 20464976 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2578112'] with mtr_portfolio_ids : ['20464976'] and first list_photo_ids : [] new path : /proc/930758/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.9052400588989258 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Wed Feb 12 01:12:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3366 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-12 01:12:35.261737: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-12 01:12:35.291153: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-12 01:12:35.292751: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff5f0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-12 01:12:35.292804: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-12 01:12:35.295820: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-12 01:12:35.560748: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x226bed50 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-12 01:12:35.560814: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-12 01:12:35.561884: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 01:12:35.562666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 01:12:35.567136: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 01:12:35.570291: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 01:12:35.571942: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 01:12:35.574214: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 01:12:35.575282: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 01:12:35.581075: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 01:12:35.582159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 01:12:35.582239: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 01:12:35.582811: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-12 01:12:35.582829: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-12 01:12:35.582839: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-12 01:12:35.583792: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2914 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-12 01:12:35.903894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 01:12:35.904050: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 01:12:35.904080: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 01:12:35.904106: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 01:12:35.904130: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 01:12:35.904155: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 01:12:35.904178: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 01:12:35.904202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 01:12:35.905098: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 01:12:35.906122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-12 01:12:35.906151: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-12 01:12:35.906165: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 01:12:35.906177: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-12 01:12:35.906190: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-12 01:12:35.906202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-12 01:12:35.906214: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-12 01:12:35.906227: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 01:12:35.906994: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-12 01:12:35.907036: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-12 01:12:35.907045: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-12 01:12:35.907053: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-12 01:12:35.907928: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2914 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-12 01:12:47.324397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-12 01:12:47.494127: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-12 01:12:49.054188: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.054798: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.11G (2268581120 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.054836: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.055505: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.055528: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.062807: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.062830: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.063485: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.063510: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.070155: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.070185: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.070782: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.070799: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.102226: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.102257: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.102874: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.102892: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.108829: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.108853: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.109468: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.109489: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-12 01:12:49.143960: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.144579: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.146479: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.147110: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.189813: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.190446: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.192440: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.193037: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.218584: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.219204: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.220781: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.221377: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.226871: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.227544: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.229285: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.229919: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.235736: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.236345: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.237929: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.238535: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.264971: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.265555: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.266150: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.266724: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.270140: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.270712: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.285989: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.286575: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.287167: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.287741: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.299936: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.300531: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.301102: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.301671: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.305840: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.306415: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.310983: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.311582: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.323488: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.324062: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.328167: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.328737: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.329304: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.329869: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.330617: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.331280: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.341836: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.342408: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.343004: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.343575: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.344141: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.344706: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.359348: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.359923: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.402296: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.402916: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.403509: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.404087: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.411940: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.412541: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.428415: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.429301: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.430163: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.431038: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.435095: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.435678: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.436257: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.436956: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.437938: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.437956: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-12 01:12:49.448068: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.448649: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.457540: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.458166: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.458765: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.459353: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.459934: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-12 01:12:49.460542: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.35G (2520645632 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 11 Detection mask done ! Trying to reset tf kernel 931426 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9798 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10998 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.00038170814514160156 nb_pixel_total : 6478 time to create 1 rle with old method : 0.008723974227905273 length of segment : 99 time for calcul the mask position with numpy : 0.002516508102416992 nb_pixel_total : 68946 time to create 1 rle with old method : 0.08627200126647949 length of segment : 341 time for calcul the mask position with numpy : 0.29244470596313477 nb_pixel_total : 298582 time to create 1 rle with new method : 0.1660001277923584 length of segment : 627 time for calcul the mask position with numpy : 0.00018978118896484375 nb_pixel_total : 6414 time to create 1 rle with old method : 0.008353948593139648 length of segment : 85 time for calcul the mask position with numpy : 0.0017082691192626953 nb_pixel_total : 109699 time to create 1 rle with old method : 0.11681103706359863 length of segment : 435 time for calcul the mask position with numpy : 0.0050907135009765625 nb_pixel_total : 316883 time to create 1 rle with new method : 0.027248144149780273 length of segment : 688 time for calcul the mask position with numpy : 0.0007658004760742188 nb_pixel_total : 37728 time to create 1 rle with old method : 0.04145002365112305 length of segment : 453 time for calcul the mask position with numpy : 0.002299070358276367 nb_pixel_total : 151355 time to create 1 rle with new method : 0.007880687713623047 length of segment : 613 time for calcul the mask position with numpy : 0.0003924369812011719 nb_pixel_total : 16948 time to create 1 rle with old method : 0.018391847610473633 length of segment : 170 time for calcul the mask position with numpy : 0.0002307891845703125 nb_pixel_total : 5354 time to create 1 rle with old method : 0.006074666976928711 length of segment : 181 time for calcul the mask position with numpy : 0.0007171630859375 nb_pixel_total : 39296 time to create 1 rle with old method : 0.042862892150878906 length of segment : 353 time for calcul the mask position with numpy : 0.00022840499877929688 nb_pixel_total : 11672 time to create 1 rle with old method : 0.013643980026245117 length of segment : 103 time for calcul the mask position with numpy : 0.00010061264038085938 nb_pixel_total : 2760 time to create 1 rle with old method : 0.0034596920013427734 length of segment : 63 time for calcul the mask position with numpy : 0.00043082237243652344 nb_pixel_total : 21685 time to create 1 rle with old method : 0.029406309127807617 length of segment : 226 time for calcul the mask position with numpy : 0.0007865428924560547 nb_pixel_total : 36544 time to create 1 rle with old method : 0.041843414306640625 length of segment : 337 time for calcul the mask position with numpy : 0.0002689361572265625 nb_pixel_total : 7306 time to create 1 rle with old method : 0.008633613586425781 length of segment : 115 time for calcul the mask position with numpy : 0.00182342529296875 nb_pixel_total : 93948 time to create 1 rle with old method : 0.11470890045166016 length of segment : 617 time for calcul the mask position with numpy : 0.0009620189666748047 nb_pixel_total : 16456 time to create 1 rle with old method : 0.01955580711364746 length of segment : 143 time for calcul the mask position with numpy : 0.001955270767211914 nb_pixel_total : 58211 time to create 1 rle with old method : 0.06817150115966797 length of segment : 287 time for calcul the mask position with numpy : 0.0009043216705322266 nb_pixel_total : 21531 time to create 1 rle with old method : 0.024116039276123047 length of segment : 169 time for calcul the mask position with numpy : 0.0002257823944091797 nb_pixel_total : 4289 time to create 1 rle with old method : 0.005316495895385742 length of segment : 57 time for calcul the mask position with numpy : 0.0004668235778808594 nb_pixel_total : 10691 time to create 1 rle with old method : 0.012186050415039062 length of segment : 99 time for calcul the mask position with numpy : 0.002307415008544922 nb_pixel_total : 63185 time to create 1 rle with old method : 0.07440996170043945 length of segment : 443 time for calcul the mask position with numpy : 0.0005106925964355469 nb_pixel_total : 10629 time to create 1 rle with old method : 0.01296377182006836 length of segment : 135 time for calcul the mask position with numpy : 0.00045800209045410156 nb_pixel_total : 9715 time to create 1 rle with old method : 0.01592230796813965 length of segment : 102 time for calcul the mask position with numpy : 0.00045371055603027344 nb_pixel_total : 4976 time to create 1 rle with old method : 0.008352279663085938 length of segment : 148 time for calcul the mask position with numpy : 0.00024437904357910156 nb_pixel_total : 2794 time to create 1 rle with old method : 0.00470280647277832 length of segment : 70 time for calcul the mask position with numpy : 0.0052471160888671875 nb_pixel_total : 190225 time to create 1 rle with new method : 0.00915980339050293 length of segment : 442 time for calcul the mask position with numpy : 0.0017213821411132812 nb_pixel_total : 52779 time to create 1 rle with old method : 0.05675315856933594 length of segment : 358 time for calcul the mask position with numpy : 0.0015530586242675781 nb_pixel_total : 56155 time to create 1 rle with old method : 0.06343817710876465 length of segment : 268 time for calcul the mask position with numpy : 0.0016016960144042969 nb_pixel_total : 34588 time to create 1 rle with old method : 0.0390772819519043 length of segment : 202 time for calcul the mask position with numpy : 0.0003521442413330078 nb_pixel_total : 9699 time to create 1 rle with old method : 0.011914491653442383 length of segment : 110 time for calcul the mask position with numpy : 0.0025980472564697266 nb_pixel_total : 69507 time to create 1 rle with old method : 0.08386898040771484 length of segment : 440 time for calcul the mask position with numpy : 0.0001842975616455078 nb_pixel_total : 3753 time to create 1 rle with old method : 0.004774570465087891 length of segment : 62 time for calcul the mask position with numpy : 0.0019199848175048828 nb_pixel_total : 32969 time to create 1 rle with old method : 0.08749628067016602 length of segment : 305 time for calcul the mask position with numpy : 0.0005602836608886719 nb_pixel_total : 10889 time to create 1 rle with old method : 0.01282048225402832 length of segment : 140 time for calcul the mask position with numpy : 0.0017998218536376953 nb_pixel_total : 25085 time to create 1 rle with old method : 0.02849435806274414 length of segment : 291 time for calcul the mask position with numpy : 0.0001766681671142578 nb_pixel_total : 2536 time to create 1 rle with old method : 0.002913236618041992 length of segment : 57 time for calcul the mask position with numpy : 0.0011458396911621094 nb_pixel_total : 43052 time to create 1 rle with old method : 0.04752850532531738 length of segment : 421 time for calcul the mask position with numpy : 0.00031685829162597656 nb_pixel_total : 6171 time to create 1 rle with old method : 0.007013797760009766 length of segment : 74 time for calcul the mask position with numpy : 0.01542806625366211 nb_pixel_total : 353009 time to create 1 rle with new method : 0.0272674560546875 length of segment : 658 time for calcul the mask position with numpy : 0.002448558807373047 nb_pixel_total : 77571 time to create 1 rle with old method : 0.0846104621887207 length of segment : 516 time for calcul the mask position with numpy : 0.00047850608825683594 nb_pixel_total : 18640 time to create 1 rle with old method : 0.021580219268798828 length of segment : 312 time for calcul the mask position with numpy : 0.0036704540252685547 nb_pixel_total : 154338 time to create 1 rle with new method : 0.008052587509155273 length of segment : 646 time for calcul the mask position with numpy : 0.0015311241149902344 nb_pixel_total : 45181 time to create 1 rle with old method : 0.05763721466064453 length of segment : 290 time for calcul the mask position with numpy : 0.22487545013427734 nb_pixel_total : 1916005 time to create 1 rle with new method : 0.3708229064941406 length of segment : 2617 time for calcul the mask position with numpy : 0.0005717277526855469 nb_pixel_total : 27204 time to create 1 rle with old method : 0.02965092658996582 length of segment : 145 time for calcul the mask position with numpy : 0.0003688335418701172 nb_pixel_total : 19921 time to create 1 rle with old method : 0.025269508361816406 length of segment : 158 time for calcul the mask position with numpy : 0.0005438327789306641 nb_pixel_total : 21338 time to create 1 rle with old method : 0.03508639335632324 length of segment : 157 time for calcul the mask position with numpy : 0.001556396484375 nb_pixel_total : 70468 time to create 1 rle with old method : 0.07711076736450195 length of segment : 473 time for calcul the mask position with numpy : 0.0003867149353027344 nb_pixel_total : 20644 time to create 1 rle with old method : 0.025391340255737305 length of segment : 162 time for calcul the mask position with numpy : 0.00016498565673828125 nb_pixel_total : 1848 time to create 1 rle with old method : 0.0022771358489990234 length of segment : 72 time for calcul the mask position with numpy : 0.008998870849609375 nb_pixel_total : 370601 time to create 1 rle with new method : 0.01449275016784668 length of segment : 506 time for calcul the mask position with numpy : 0.001207113265991211 nb_pixel_total : 72808 time to create 1 rle with old method : 0.08317041397094727 length of segment : 469 time for calcul the mask position with numpy : 0.0004639625549316406 nb_pixel_total : 29810 time to create 1 rle with old method : 0.03116321563720703 length of segment : 224 time for calcul the mask position with numpy : 0.0011229515075683594 nb_pixel_total : 46787 time to create 1 rle with old method : 0.05411219596862793 length of segment : 309 time for calcul the mask position with numpy : 0.23399066925048828 nb_pixel_total : 3836567 time to create 1 rle with new method : 0.23983144760131836 length of segment : 3419 time for calcul the mask position with numpy : 0.002405405044555664 nb_pixel_total : 75806 time to create 1 rle with old method : 0.0814363956451416 length of segment : 653 time for calcul the mask position with numpy : 0.00015592575073242188 nb_pixel_total : 2585 time to create 1 rle with old method : 0.002992391586303711 length of segment : 64 time for calcul the mask position with numpy : 0.0015251636505126953 nb_pixel_total : 100633 time to create 1 rle with old method : 0.10935235023498535 length of segment : 374 time for calcul the mask position with numpy : 0.0004889965057373047 nb_pixel_total : 15470 time to create 1 rle with old method : 0.017163515090942383 length of segment : 173 time for calcul the mask position with numpy : 0.0018062591552734375 nb_pixel_total : 67725 time to create 1 rle with old method : 0.07445478439331055 length of segment : 303 time for calcul the mask position with numpy : 0.0009989738464355469 nb_pixel_total : 32353 time to create 1 rle with old method : 0.03686714172363281 length of segment : 238 time for calcul the mask position with numpy : 0.0002760887145996094 nb_pixel_total : 8050 time to create 1 rle with old method : 0.00921940803527832 length of segment : 61 time for calcul the mask position with numpy : 0.0017104148864746094 nb_pixel_total : 69640 time to create 1 rle with old method : 0.07533645629882812 length of segment : 370 time for calcul the mask position with numpy : 0.0001900196075439453 nb_pixel_total : 2814 time to create 1 rle with old method : 0.0037605762481689453 length of segment : 89 time for calcul the mask position with numpy : 0.0002758502960205078 nb_pixel_total : 6287 time to create 1 rle with old method : 0.0074689388275146484 length of segment : 98 time for calcul the mask position with numpy : 0.00023031234741210938 nb_pixel_total : 2297 time to create 1 rle with old method : 0.002712249755859375 length of segment : 109 time for calcul the mask position with numpy : 0.002658843994140625 nb_pixel_total : 86847 time to create 1 rle with old method : 0.12170934677124023 length of segment : 450 time for calcul the mask position with numpy : 0.0005972385406494141 nb_pixel_total : 18170 time to create 1 rle with old method : 0.019763946533203125 length of segment : 164 time for calcul the mask position with numpy : 0.018236875534057617 nb_pixel_total : 201239 time to create 1 rle with new method : 0.08003616333007812 length of segment : 820 time for calcul the mask position with numpy : 0.001085042953491211 nb_pixel_total : 38438 time to create 1 rle with old method : 0.041910648345947266 length of segment : 230 time for calcul the mask position with numpy : 0.0005919933319091797 nb_pixel_total : 38722 time to create 1 rle with old method : 0.04260683059692383 length of segment : 225 time for calcul the mask position with numpy : 0.002519369125366211 nb_pixel_total : 112047 time to create 1 rle with old method : 0.12160158157348633 length of segment : 288 time for calcul the mask position with numpy : 0.0005762577056884766 nb_pixel_total : 9084 time to create 1 rle with old method : 0.010401010513305664 length of segment : 221 time for calcul the mask position with numpy : 0.007342100143432617 nb_pixel_total : 248432 time to create 1 rle with new method : 0.025702238082885742 length of segment : 799 time for calcul the mask position with numpy : 0.0006053447723388672 nb_pixel_total : 12717 time to create 1 rle with old method : 0.014269828796386719 length of segment : 146 time for calcul the mask position with numpy : 0.0003669261932373047 nb_pixel_total : 8215 time to create 1 rle with old method : 0.00934743881225586 length of segment : 124 time for calcul the mask position with numpy : 0.0005679130554199219 nb_pixel_total : 12642 time to create 1 rle with old method : 0.014493942260742188 length of segment : 167 time for calcul the mask position with numpy : 0.00074005126953125 nb_pixel_total : 12413 time to create 1 rle with old method : 0.01421809196472168 length of segment : 135 time for calcul the mask position with numpy : 0.001386404037475586 nb_pixel_total : 38956 time to create 1 rle with old method : 0.04292702674865723 length of segment : 199 time for calcul the mask position with numpy : 0.0006132125854492188 nb_pixel_total : 15424 time to create 1 rle with old method : 0.017532825469970703 length of segment : 151 time for calcul the mask position with numpy : 0.004645109176635742 nb_pixel_total : 189353 time to create 1 rle with new method : 0.007732868194580078 length of segment : 673 time for calcul the mask position with numpy : 0.0022737979888916016 nb_pixel_total : 47104 time to create 1 rle with old method : 0.05417919158935547 length of segment : 312 time for calcul the mask position with numpy : 0.0001285076141357422 nb_pixel_total : 3259 time to create 1 rle with old method : 0.004031181335449219 length of segment : 60 time for calcul the mask position with numpy : 0.0034537315368652344 nb_pixel_total : 102283 time to create 1 rle with old method : 0.11218833923339844 length of segment : 353 time for calcul the mask position with numpy : 0.000274658203125 nb_pixel_total : 5975 time to create 1 rle with old method : 0.00716090202331543 length of segment : 155 time for calcul the mask position with numpy : 0.0004534721374511719 nb_pixel_total : 7286 time to create 1 rle with old method : 0.00844264030456543 length of segment : 118 time for calcul the mask position with numpy : 0.0002658367156982422 nb_pixel_total : 3411 time to create 1 rle with old method : 0.003968477249145508 length of segment : 87 time for calcul the mask position with numpy : 0.0015194416046142578 nb_pixel_total : 53367 time to create 1 rle with old method : 0.06137251853942871 length of segment : 278 time for calcul the mask position with numpy : 0.0015845298767089844 nb_pixel_total : 51029 time to create 1 rle with old method : 0.0763397216796875 length of segment : 697 time for calcul the mask position with numpy : 0.00042700767517089844 nb_pixel_total : 1253 time to create 1 rle with old method : 0.0016312599182128906 length of segment : 105 time for calcul the mask position with numpy : 0.0016410350799560547 nb_pixel_total : 98526 time to create 1 rle with old method : 0.10288119316101074 length of segment : 346 time for calcul the mask position with numpy : 0.002026081085205078 nb_pixel_total : 64238 time to create 1 rle with old method : 0.06682395935058594 length of segment : 362 time for calcul the mask position with numpy : 0.00042724609375 nb_pixel_total : 9268 time to create 1 rle with old method : 0.010243892669677734 length of segment : 119 time for calcul the mask position with numpy : 0.0009253025054931641 nb_pixel_total : 21222 time to create 1 rle with old method : 0.023137331008911133 length of segment : 244 time for calcul the mask position with numpy : 0.0009658336639404297 nb_pixel_total : 30277 time to create 1 rle with old method : 0.03302431106567383 length of segment : 210 time for calcul the mask position with numpy : 0.00014472007751464844 nb_pixel_total : 1656 time to create 1 rle with old method : 0.0018246173858642578 length of segment : 56 time for calcul the mask position with numpy : 0.04679131507873535 nb_pixel_total : 1502323 time to create 1 rle with new method : 0.25165438652038574 length of segment : 2367 time for calcul the mask position with numpy : 0.003392457962036133 nb_pixel_total : 152830 time to create 1 rle with new method : 0.005541563034057617 length of segment : 407 time for calcul the mask position with numpy : 0.0002491474151611328 nb_pixel_total : 6221 time to create 1 rle with old method : 0.007319450378417969 length of segment : 61 time for calcul the mask position with numpy : 0.0014500617980957031 nb_pixel_total : 53856 time to create 1 rle with old method : 0.058411598205566406 length of segment : 333 time for calcul the mask position with numpy : 0.0003826618194580078 nb_pixel_total : 9420 time to create 1 rle with old method : 0.010303974151611328 length of segment : 129 time for calcul the mask position with numpy : 9.107589721679688e-05 nb_pixel_total : 2229 time to create 1 rle with old method : 0.0027153491973876953 length of segment : 66 time for calcul the mask position with numpy : 0.002619504928588867 nb_pixel_total : 118443 time to create 1 rle with old method : 0.1269223690032959 length of segment : 626 time for calcul the mask position with numpy : 0.00022268295288085938 nb_pixel_total : 4702 time to create 1 rle with old method : 0.005646467208862305 length of segment : 46 time for calcul the mask position with numpy : 0.0018243789672851562 nb_pixel_total : 82910 time to create 1 rle with old method : 0.08959221839904785 length of segment : 479 time for calcul the mask position with numpy : 0.00021314620971679688 nb_pixel_total : 3888 time to create 1 rle with old method : 0.004756927490234375 length of segment : 116 time for calcul the mask position with numpy : 0.0015130043029785156 nb_pixel_total : 51297 time to create 1 rle with old method : 0.055622100830078125 length of segment : 251 time for calcul the mask position with numpy : 0.001804351806640625 nb_pixel_total : 95620 time to create 1 rle with old method : 0.10041356086730957 length of segment : 429 time for calcul the mask position with numpy : 0.0024955272674560547 nb_pixel_total : 121253 time to create 1 rle with old method : 0.13654446601867676 length of segment : 645 time for calcul the mask position with numpy : 0.0022869110107421875 nb_pixel_total : 123417 time to create 1 rle with old method : 0.13254022598266602 length of segment : 722 time for calcul the mask position with numpy : 0.0025458335876464844 nb_pixel_total : 114991 time to create 1 rle with old method : 0.12279605865478516 length of segment : 383 time for calcul the mask position with numpy : 0.0010704994201660156 nb_pixel_total : 57859 time to create 1 rle with old method : 0.06048464775085449 length of segment : 504 time for calcul the mask position with numpy : 0.00021648406982421875 nb_pixel_total : 3057 time to create 1 rle with old method : 0.003625154495239258 length of segment : 101 time for calcul the mask position with numpy : 0.010254859924316406 nb_pixel_total : 535399 time to create 1 rle with new method : 0.019870996475219727 length of segment : 1549 time for calcul the mask position with numpy : 0.0019588470458984375 nb_pixel_total : 54239 time to create 1 rle with old method : 0.055478811264038086 length of segment : 367 time for calcul the mask position with numpy : 0.0025832653045654297 nb_pixel_total : 56556 time to create 1 rle with old method : 0.060315847396850586 length of segment : 427 time for calcul the mask position with numpy : 0.002435445785522461 nb_pixel_total : 71706 time to create 1 rle with old method : 0.0745704174041748 length of segment : 381 time for calcul the mask position with numpy : 0.0006742477416992188 nb_pixel_total : 20300 time to create 1 rle with old method : 0.021435976028442383 length of segment : 184 time for calcul the mask position with numpy : 0.00011682510375976562 nb_pixel_total : 2879 time to create 1 rle with old method : 0.0031790733337402344 length of segment : 88 time for calcul the mask position with numpy : 0.0011184215545654297 nb_pixel_total : 51378 time to create 1 rle with old method : 0.05381941795349121 length of segment : 421 time for calcul the mask position with numpy : 0.003678560256958008 nb_pixel_total : 137535 time to create 1 rle with old method : 0.1441190242767334 length of segment : 580 time for calcul the mask position with numpy : 0.0016117095947265625 nb_pixel_total : 46402 time to create 1 rle with old method : 0.04985690116882324 length of segment : 315 time for calcul the mask position with numpy : 0.0010504722595214844 nb_pixel_total : 22341 time to create 1 rle with old method : 0.02421855926513672 length of segment : 332 time for calcul the mask position with numpy : 0.00036716461181640625 nb_pixel_total : 9545 time to create 1 rle with old method : 0.010888338088989258 length of segment : 104 time for calcul the mask position with numpy : 0.0007426738739013672 nb_pixel_total : 21056 time to create 1 rle with old method : 0.02361464500427246 length of segment : 220 time for calcul the mask position with numpy : 0.0007834434509277344 nb_pixel_total : 25647 time to create 1 rle with old method : 0.02911663055419922 length of segment : 283 time for calcul the mask position with numpy : 0.0003135204315185547 nb_pixel_total : 17018 time to create 1 rle with old method : 0.019726991653442383 length of segment : 173 time for calcul the mask position with numpy : 0.00033593177795410156 nb_pixel_total : 8486 time to create 1 rle with old method : 0.009987592697143555 length of segment : 136 time for calcul the mask position with numpy : 0.0005414485931396484 nb_pixel_total : 11339 time to create 1 rle with old method : 0.013060808181762695 length of segment : 149 time for calcul the mask position with numpy : 0.0002636909484863281 nb_pixel_total : 9735 time to create 1 rle with old method : 0.010849237442016602 length of segment : 170 time for calcul the mask position with numpy : 0.0005040168762207031 nb_pixel_total : 11868 time to create 1 rle with old method : 0.013302803039550781 length of segment : 196 time for calcul the mask position with numpy : 0.00470733642578125 nb_pixel_total : 144844 time to create 1 rle with old method : 0.15314173698425293 length of segment : 612 time for calcul the mask position with numpy : 0.00017786026000976562 nb_pixel_total : 2542 time to create 1 rle with old method : 0.002975940704345703 length of segment : 64 time for calcul the mask position with numpy : 0.00025177001953125 nb_pixel_total : 5112 time to create 1 rle with old method : 0.005912065505981445 length of segment : 93 time for calcul the mask position with numpy : 0.0003848075866699219 nb_pixel_total : 14241 time to create 1 rle with old method : 0.01579761505126953 length of segment : 117 time for calcul the mask position with numpy : 0.0030488967895507812 nb_pixel_total : 121904 time to create 1 rle with old method : 0.1293621063232422 length of segment : 387 time for calcul the mask position with numpy : 0.0003666877746582031 nb_pixel_total : 10450 time to create 1 rle with old method : 0.011907577514648438 length of segment : 115 time for calcul the mask position with numpy : 0.000232696533203125 nb_pixel_total : 8575 time to create 1 rle with old method : 0.009477376937866211 length of segment : 154 time for calcul the mask position with numpy : 0.010296821594238281 nb_pixel_total : 378894 time to create 1 rle with new method : 0.01730656623840332 length of segment : 613 time for calcul the mask position with numpy : 0.001489877700805664 nb_pixel_total : 59180 time to create 1 rle with old method : 0.063446044921875 length of segment : 298 time for calcul the mask position with numpy : 0.0027379989624023438 nb_pixel_total : 122237 time to create 1 rle with old method : 0.12978768348693848 length of segment : 462 time for calcul the mask position with numpy : 0.0015099048614501953 nb_pixel_total : 29523 time to create 1 rle with old method : 0.034632205963134766 length of segment : 322 time for calcul the mask position with numpy : 0.00019860267639160156 nb_pixel_total : 5939 time to create 1 rle with old method : 0.0070362091064453125 length of segment : 85 time for calcul the mask position with numpy : 0.0001232624053955078 nb_pixel_total : 2001 time to create 1 rle with old method : 0.0023322105407714844 length of segment : 44 time for calcul the mask position with numpy : 0.0001862049102783203 nb_pixel_total : 5940 time to create 1 rle with old method : 0.006975412368774414 length of segment : 109 time for calcul the mask position with numpy : 0.0016925334930419922 nb_pixel_total : 115319 time to create 1 rle with old method : 0.1255786418914795 length of segment : 369 time for calcul the mask position with numpy : 0.001432657241821289 nb_pixel_total : 55309 time to create 1 rle with old method : 0.06289792060852051 length of segment : 256 time for calcul the mask position with numpy : 0.00024890899658203125 nb_pixel_total : 6521 time to create 1 rle with old method : 0.007609844207763672 length of segment : 87 time for calcul the mask position with numpy : 0.0016908645629882812 nb_pixel_total : 45714 time to create 1 rle with old method : 0.05141901969909668 length of segment : 379 time for calcul the mask position with numpy : 0.00029277801513671875 nb_pixel_total : 11330 time to create 1 rle with old method : 0.013258934020996094 length of segment : 130 time for calcul the mask position with numpy : 0.003823518753051758 nb_pixel_total : 169191 time to create 1 rle with new method : 0.009265899658203125 length of segment : 945 time for calcul the mask position with numpy : 0.0003821849822998047 nb_pixel_total : 11055 time to create 1 rle with old method : 0.012561559677124023 length of segment : 189 time for calcul the mask position with numpy : 0.0005178451538085938 nb_pixel_total : 11389 time to create 1 rle with old method : 0.012027740478515625 length of segment : 223 time for calcul the mask position with numpy : 0.0002777576446533203 nb_pixel_total : 7768 time to create 1 rle with old method : 0.009068489074707031 length of segment : 70 time for calcul the mask position with numpy : 0.002985239028930664 nb_pixel_total : 71109 time to create 1 rle with old method : 0.07571005821228027 length of segment : 630 time for calcul the mask position with numpy : 0.002763032913208008 nb_pixel_total : 107754 time to create 1 rle with old method : 0.11035370826721191 length of segment : 361 time for calcul the mask position with numpy : 0.0013036727905273438 nb_pixel_total : 45328 time to create 1 rle with old method : 0.049430131912231445 length of segment : 488 time for calcul the mask position with numpy : 0.002246379852294922 nb_pixel_total : 26945 time to create 1 rle with old method : 0.030323266983032227 length of segment : 303 time for calcul the mask position with numpy : 0.0001304149627685547 nb_pixel_total : 1934 time to create 1 rle with old method : 0.002125978469848633 length of segment : 56 time for calcul the mask position with numpy : 0.0006303787231445312 nb_pixel_total : 13585 time to create 1 rle with old method : 0.014542818069458008 length of segment : 229 time spent for convertir_results : 17.7063045501709 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 172 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 54218 save missing photos in datou_result : time spend for datou_step_exec : 70.63553404808044 time spend to save output : 6.228373765945435 total time spend for step 1 : 76.86390781402588 step2:crop_condition Wed Feb 12 01:13:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 172 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 49 About to insert : list_path_to_insert length 49 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1336797885, 1336797871, 1336797806, 1336797703, 1336796211, 1336796103, 1336794936, 1336794898, 1336794754, 1336794688] Looping around the photos to save general results len do output : 96 /-3670629389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670883175Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670883181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670883174Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670883173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670883176Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670883177Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670883179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629513Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3670629503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797885', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797871', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797806', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797703', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796211', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796103', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794936', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794898', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794754', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794688', None, None, None, None, None, '2578112') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 298 time used for this insertion : 0.1454911231994629 save_final save missing photos in datou_result : time spend for datou_step_exec : 35.582772970199585 time spend to save output : 0.14844727516174316 total time spend for step 2 : 35.73122024536133 step3:thcl Wed Feb 12 01:14:24 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.011223793029785156 time to convert the images to numpy array : 0.10736608505249023 time to import caffe and check if the image exist : 0.013763427734375 time to convert the images to numpy array : 0.1307995319366455 time to import caffe and check if the image exist : 0.01301264762878418 time to convert the images to numpy array : 0.13988208770751953 time to import caffe and check if the image exist : 0.020348310470581055 time to convert the images to numpy array : 0.13986659049987793 time to import caffe and check if the image exist : 0.013422250747680664 time to convert the images to numpy array : 0.14573335647583008 time to import caffe and check if the image exist : 0.017128467559814453 time to convert the images to numpy array : 0.14806914329528809 time to import caffe and check if the image exist : 0.018045663833618164 time to convert the images to numpy array : 0.1590595245361328 time to import caffe and check if the image exist : 0.01374506950378418 time to convert the images to numpy array : 0.1694176197052002 time to import caffe and check if the image exist : 0.015551328659057617 time to convert the images to numpy array : 0.17171335220336914 time to import caffe and check if the image exist : 0.02014017105102539 time to convert the images to numpy array : 0.17144107818603516 total time to convert the images to numpy array : 0.629859447479248 list photo_ids error: [] list photo_ids correct : [-3670629519, -3670629370, -3670629372, -3670629390, -3670629393, -3670629396, -3670629453, -3670629463, -3670629490, -3670629500, -3670629389, -3670629384, -3670629391, -3670629409, -3670629421, -3670629431, -3670629443, -3670629483, -3670629478, -3670629488, -3670629492, -3670629485, -3670883174, -3670629371, -3670629375, -3670883173, -3670629380, -3670629378, -3670629376, -3670629385, -3670629381, -3670629368, -3670629398, -3670629402, -3670883175, -3670629426, -3670629458, -3670629451, -3670629452, -3670629456, -3670629502, -3670629507, -3670629517, -3670629437, -3670629442, -3670629503, -3670629470, -3670629464, -3670629475, -3670883181, -3670629506, -3670629400, -3670629417, -3670629434, -3670629472, -3670629486, -3670629404, -3670629403, -3670629394, -3670629392, -3670629406, -3670629414, -3670629415, -3670629412, -3670629405, -3670629408, -3670883176, -3670883177, -3670629419, -3670629424, -3670629433, -3670629430, -3670629435, -3670629444, -3670883179, -3670629447, -3670629481, -3670629480, -3670629515, -3670629511, -3670629510, -3670629505, -3670629504, -3670629513, -3670629520, -3670629522, -3670629455, -3670629474, -3670629461, -3670629473, -3670629468, -3670629471, -3670629484, -3670629489, -3670629494, -3670629479] number of photos to traite : 96 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10776 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.1087517738342285 time used to do the prediction : 0.2766916751861572 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.1139657497406006 time used to do the prediction : 0.17029619216918945 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 8.821487426757812e-06 save missing photos in datou_result : time spend for datou_step_exec : 10.3596031665802 time spend to save output : 0.0006341934204101562 total time spend for step 3 : 10.36023736000061 step4:merge_mask_thcl_custom Wed Feb 12 01:14:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 172 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1336797885, 1336797871, 1336797806, 1336797703, 1336796211, 1336796103, 1336794936, 1336794898, 1336794754, 1336794688] Looping around the photos to save general results len do output : 10 /1336797885Didn't retrieve data .Didn't retrieve data . /1336797871Didn't retrieve data .Didn't retrieve data . /1336797806Didn't retrieve data .Didn't retrieve data . /1336797703Didn't retrieve data .Didn't retrieve data . /1336796211Didn't retrieve data .Didn't retrieve data . /1336796103Didn't retrieve data .Didn't retrieve data . /1336794936Didn't retrieve data .Didn't retrieve data . /1336794898Didn't retrieve data .Didn't retrieve data . /1336794754Didn't retrieve data .Didn't retrieve data . /1336794688Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797885', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797871', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797806', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797703', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796211', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796103', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794936', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794898', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794754', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794688', None, None, None, None, None, '2578112') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.015801668167114258 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06133270263671875 time spend to save output : 0.016213178634643555 total time spend for step 4 : 0.0775458812713623 step5:rle_unique_nms_with_priority Wed Feb 12 01:14:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 21 nb_hashtags : 6 time to prepare the origin masks : 10.965534210205078 time for calcul the mask position with numpy : 0.4250786304473877 nb_pixel_total : 5768485 time to create 1 rle with new method : 0.4322042465209961 time for calcul the mask position with numpy : 0.0205996036529541 nb_pixel_total : 9970 time to create 1 rle with old method : 0.01056814193725586 time for calcul the mask position with numpy : 0.02111220359802246 nb_pixel_total : 39186 time to create 1 rle with old method : 0.04091596603393555 time for calcul the mask position with numpy : 0.021797895431518555 nb_pixel_total : 1385 time to create 1 rle with old method : 0.001638650894165039 time for calcul the mask position with numpy : 0.021479368209838867 nb_pixel_total : 109661 time to create 1 rle with old method : 0.11358261108398438 time for calcul the mask position with numpy : 0.02204442024230957 nb_pixel_total : 58081 time to create 1 rle with old method : 0.06178116798400879 time for calcul the mask position with numpy : 0.01961541175842285 nb_pixel_total : 4256 time to create 1 rle with old method : 0.004668712615966797 time for calcul the mask position with numpy : 0.020851850509643555 nb_pixel_total : 16306 time to create 1 rle with old method : 0.016378402709960938 time for calcul the mask position with numpy : 0.019930124282836914 nb_pixel_total : 7173 time to create 1 rle with old method : 0.00774693489074707 time for calcul the mask position with numpy : 0.020162105560302734 nb_pixel_total : 16829 time to create 1 rle with old method : 0.017066478729248047 time for calcul the mask position with numpy : 0.021172285079956055 nb_pixel_total : 2734 time to create 1 rle with old method : 0.003047943115234375 time for calcul the mask position with numpy : 0.019589662551879883 nb_pixel_total : 5355 time to create 1 rle with old method : 0.006064891815185547 time for calcul the mask position with numpy : 0.01950240135192871 nb_pixel_total : 36481 time to create 1 rle with old method : 0.03708291053771973 time for calcul the mask position with numpy : 0.02164602279663086 nb_pixel_total : 151040 time to create 1 rle with new method : 0.46228742599487305 time for calcul the mask position with numpy : 0.021462440490722656 nb_pixel_total : 11626 time to create 1 rle with old method : 0.01256251335144043 time for calcul the mask position with numpy : 0.022812604904174805 nb_pixel_total : 315994 time to create 1 rle with new method : 0.6507947444915771 time for calcul the mask position with numpy : 0.0205838680267334 nb_pixel_total : 21719 time to create 1 rle with old method : 0.023361682891845703 time for calcul the mask position with numpy : 0.020641326904296875 nb_pixel_total : 6368 time to create 1 rle with old method : 0.006686210632324219 time for calcul the mask position with numpy : 0.0207061767578125 nb_pixel_total : 68594 time to create 1 rle with old method : 0.06899070739746094 time for calcul the mask position with numpy : 0.022640228271484375 nb_pixel_total : 298176 time to create 1 rle with new method : 0.5170931816101074 time for calcul the mask position with numpy : 0.021076679229736328 nb_pixel_total : 6439 time to create 1 rle with old method : 0.00703120231628418 time for calcul the mask position with numpy : 0.02131366729736328 nb_pixel_total : 94382 time to create 1 rle with old method : 0.0985724925994873 create new chi : 3.5665578842163086 time to delete rle : 0.016517162322998047 batch 1 Loaded 24 chid ids of type : 3726 Number RLEs to save : 13464 TO DO : save crop sub photo not yet done ! save time : 0.9542982578277588 nb_obj : 22 nb_hashtags : 6 time to prepare the origin masks : 10.812661170959473 time for calcul the mask position with numpy : 0.4619905948638916 nb_pixel_total : 5999728 time to create 1 rle with new method : 0.8246095180511475 time for calcul the mask position with numpy : 0.026006698608398438 nb_pixel_total : 17502 time to create 1 rle with old method : 0.02156853675842285 time for calcul the mask position with numpy : 0.025246381759643555 nb_pixel_total : 25149 time to create 1 rle with old method : 0.03228449821472168 time for calcul the mask position with numpy : 0.025765180587768555 nb_pixel_total : 10540 time to create 1 rle with old method : 0.013878345489501953 time for calcul the mask position with numpy : 0.029688119888305664 nb_pixel_total : 52776 time to create 1 rle with old method : 0.06535196304321289 time for calcul the mask position with numpy : 0.022919178009033203 nb_pixel_total : 55930 time to create 1 rle with old method : 0.061623573303222656 time for calcul the mask position with numpy : 0.020848989486694336 nb_pixel_total : 34462 time to create 1 rle with old method : 0.037133216857910156 time for calcul the mask position with numpy : 0.021199941635131836 nb_pixel_total : 6120 time to create 1 rle with old method : 0.0065877437591552734 time for calcul the mask position with numpy : 0.021567583084106445 nb_pixel_total : 189940 time to create 1 rle with new method : 0.5195057392120361 time for calcul the mask position with numpy : 0.021152019500732422 nb_pixel_total : 2515 time to create 1 rle with old method : 0.0028781890869140625 time for calcul the mask position with numpy : 0.021965742111206055 nb_pixel_total : 62999 time to create 1 rle with old method : 0.0670320987701416 time for calcul the mask position with numpy : 0.025473356246948242 nb_pixel_total : 352232 time to create 1 rle with new method : 0.4842679500579834 time for calcul the mask position with numpy : 0.022413253784179688 nb_pixel_total : 10839 time to create 1 rle with old method : 0.013998746871948242 time for calcul the mask position with numpy : 0.024036407470703125 nb_pixel_total : 65424 time to create 1 rle with old method : 0.0717015266418457 time for calcul the mask position with numpy : 0.022118091583251953 nb_pixel_total : 3733 time to create 1 rle with old method : 0.0041429996490478516 time for calcul the mask position with numpy : 0.021958351135253906 nb_pixel_total : 9592 time to create 1 rle with old method : 0.01085042953491211 time for calcul the mask position with numpy : 0.022050857543945312 nb_pixel_total : 2873 time to create 1 rle with old method : 0.0034949779510498047 time for calcul the mask position with numpy : 0.02178668975830078 nb_pixel_total : 2788 time to create 1 rle with old method : 0.0030133724212646484 time for calcul the mask position with numpy : 0.021502971649169922 nb_pixel_total : 4932 time to create 1 rle with old method : 0.005349874496459961 time for calcul the mask position with numpy : 0.021516799926757812 nb_pixel_total : 9698 time to create 1 rle with old method : 0.010653257369995117 time for calcul the mask position with numpy : 0.023441314697265625 nb_pixel_total : 43158 time to create 1 rle with old method : 0.06736040115356445 time for calcul the mask position with numpy : 0.024204492568969727 nb_pixel_total : 10637 time to create 1 rle with old method : 0.01193857192993164 time for calcul the mask position with numpy : 0.022711753845214844 nb_pixel_total : 76673 time to create 1 rle with old method : 0.0948178768157959 create new chi : 3.5076284408569336 time to delete rle : 0.001142263412475586 batch 1 Loaded 23 chid ids of type : 3726 Number RLEs to save : 12834 TO DO : save crop sub photo not yet done ! save time : 1.4788846969604492 nb_obj : 13 nb_hashtags : 4 time to prepare the origin masks : 6.639050483703613 time for calcul the mask position with numpy : 0.504425048828125 nb_pixel_total : 4410360 time to create 1 rle with new method : 0.691190242767334 time for calcul the mask position with numpy : 0.024976491928100586 nb_pixel_total : 29733 time to create 1 rle with old method : 0.03238677978515625 time for calcul the mask position with numpy : 0.03730034828186035 nb_pixel_total : 27122 time to create 1 rle with old method : 0.028251171112060547 time for calcul the mask position with numpy : 0.03458094596862793 nb_pixel_total : 370176 time to create 1 rle with new method : 0.5164883136749268 time for calcul the mask position with numpy : 0.03303408622741699 nb_pixel_total : 1838 time to create 1 rle with old method : 0.002221345901489258 time for calcul the mask position with numpy : 0.02030038833618164 nb_pixel_total : 13 time to create 1 rle with old method : 0.00011277198791503906 time for calcul the mask position with numpy : 0.03017449378967285 nb_pixel_total : 5325 time to create 1 rle with old method : 0.005766868591308594 time for calcul the mask position with numpy : 0.03238201141357422 nb_pixel_total : 21174 time to create 1 rle with old method : 0.022023916244506836 time for calcul the mask position with numpy : 0.032715797424316406 nb_pixel_total : 70225 time to create 1 rle with old method : 0.07277846336364746 time for calcul the mask position with numpy : 0.05143404006958008 nb_pixel_total : 1892334 time to create 1 rle with new method : 0.4614391326904297 time for calcul the mask position with numpy : 0.03312325477600098 nb_pixel_total : 19813 time to create 1 rle with old method : 0.02076864242553711 time for calcul the mask position with numpy : 0.03275322914123535 nb_pixel_total : 1541 time to create 1 rle with old method : 0.0019800662994384766 time for calcul the mask position with numpy : 0.03192782402038574 nb_pixel_total : 46690 time to create 1 rle with old method : 0.0506746768951416 time for calcul the mask position with numpy : 0.0329737663269043 nb_pixel_total : 153896 time to create 1 rle with new method : 0.47483205795288086 create new chi : 3.422931671142578 time to delete rle : 0.0018184185028076172 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 13548 TO DO : save crop sub photo not yet done ! save time : 2.9877994060516357 nb_obj : 7 nb_hashtags : 3 time to prepare the origin masks : 4.515514612197876 time for calcul the mask position with numpy : 0.19958186149597168 nb_pixel_total : 2951319 time to create 1 rle with new method : 0.373868465423584 time for calcul the mask position with numpy : 0.02102971076965332 nb_pixel_total : 15345 time to create 1 rle with old method : 0.017192840576171875 time for calcul the mask position with numpy : 0.021196603775024414 nb_pixel_total : 2571 time to create 1 rle with old method : 0.002943754196166992 time for calcul the mask position with numpy : 0.033028364181518555 nb_pixel_total : 93794 time to create 1 rle with old method : 0.11272573471069336 time for calcul the mask position with numpy : 0.041861772537231445 nb_pixel_total : 75650 time to create 1 rle with old method : 0.08129525184631348 time for calcul the mask position with numpy : 0.03486776351928711 nb_pixel_total : 51849 time to create 1 rle with old method : 0.05574512481689453 time for calcul the mask position with numpy : 0.3002891540527344 nb_pixel_total : 3827520 time to create 1 rle with new method : 0.955618143081665 time for calcul the mask position with numpy : 0.025661706924438477 nb_pixel_total : 32192 time to create 1 rle with old method : 0.03748369216918945 create new chi : 2.3848648071289062 time to delete rle : 0.0010771751403808594 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 12144 TO DO : save crop sub photo not yet done ! save time : 2.9133903980255127 nb_obj : 14 nb_hashtags : 7 time to prepare the origin masks : 8.286942958831787 time for calcul the mask position with numpy : 0.5091660022735596 nb_pixel_total : 6429733 time to create 1 rle with new method : 0.6353938579559326 time for calcul the mask position with numpy : 0.022210121154785156 nb_pixel_total : 111753 time to create 1 rle with old method : 0.13296842575073242 time for calcul the mask position with numpy : 0.022614240646362305 nb_pixel_total : 12644 time to create 1 rle with old method : 0.013655900955200195 time for calcul the mask position with numpy : 0.02087712287902832 nb_pixel_total : 8171 time to create 1 rle with old method : 0.00879669189453125 time for calcul the mask position with numpy : 0.02077960968017578 nb_pixel_total : 9031 time to create 1 rle with old method : 0.00963592529296875 time for calcul the mask position with numpy : 0.021292924880981445 nb_pixel_total : 69506 time to create 1 rle with old method : 0.08125591278076172 time for calcul the mask position with numpy : 0.022465229034423828 nb_pixel_total : 18070 time to create 1 rle with old method : 0.01971149444580078 time for calcul the mask position with numpy : 0.02228569984436035 nb_pixel_total : 795 time to create 1 rle with old method : 0.001569509506225586 time for calcul the mask position with numpy : 0.02387213706970215 nb_pixel_total : 38406 time to create 1 rle with old method : 0.04743218421936035 time for calcul the mask position with numpy : 0.022222280502319336 nb_pixel_total : 2263 time to create 1 rle with old method : 0.0025784969329833984 time for calcul the mask position with numpy : 0.022343158721923828 nb_pixel_total : 87425 time to create 1 rle with old method : 0.09481692314147949 time for calcul the mask position with numpy : 0.023619890213012695 nb_pixel_total : 6259 time to create 1 rle with old method : 0.007121562957763672 time for calcul the mask position with numpy : 0.02488088607788086 nb_pixel_total : 245398 time to create 1 rle with new method : 0.5320475101470947 time for calcul the mask position with numpy : 0.033373355865478516 nb_pixel_total : 2813 time to create 1 rle with old method : 0.0029413700103759766 time for calcul the mask position with numpy : 0.03351712226867676 nb_pixel_total : 7973 time to create 1 rle with old method : 0.008704900741577148 create new chi : 2.5141525268554688 time to delete rle : 0.0008783340454101562 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 8690 TO DO : save crop sub photo not yet done ! save time : 1.095630168914795 nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 11.963118314743042 time for calcul the mask position with numpy : 0.5009298324584961 nb_pixel_total : 6417344 time to create 1 rle with new method : 1.1446559429168701 time for calcul the mask position with numpy : 0.023409605026245117 nb_pixel_total : 64029 time to create 1 rle with old method : 0.06542444229125977 time for calcul the mask position with numpy : 0.025143146514892578 nb_pixel_total : 101921 time to create 1 rle with old method : 0.10413408279418945 time for calcul the mask position with numpy : 0.028615713119506836 nb_pixel_total : 45695 time to create 1 rle with old method : 0.04732036590576172 time for calcul the mask position with numpy : 0.02755880355834961 nb_pixel_total : 46913 time to create 1 rle with old method : 0.058690786361694336 time for calcul the mask position with numpy : 0.02563333511352539 nb_pixel_total : 9250 time to create 1 rle with old method : 0.010707378387451172 time for calcul the mask position with numpy : 0.024112462997436523 nb_pixel_total : 3383 time to create 1 rle with old method : 0.0038499832153320312 time for calcul the mask position with numpy : 0.02398204803466797 nb_pixel_total : 7251 time to create 1 rle with old method : 0.007946968078613281 time for calcul the mask position with numpy : 0.02364516258239746 nb_pixel_total : 1596 time to create 1 rle with old method : 0.0018630027770996094 time for calcul the mask position with numpy : 0.023627519607543945 nb_pixel_total : 53165 time to create 1 rle with old method : 0.05647587776184082 time for calcul the mask position with numpy : 0.025573015213012695 nb_pixel_total : 3221 time to create 1 rle with old method : 0.003683805465698242 time for calcul the mask position with numpy : 0.024343490600585938 nb_pixel_total : 15347 time to create 1 rle with old method : 0.01636505126953125 time for calcul the mask position with numpy : 0.0281369686126709 nb_pixel_total : 1249 time to create 1 rle with old method : 0.0015435218811035156 time for calcul the mask position with numpy : 0.025171518325805664 nb_pixel_total : 34855 time to create 1 rle with old method : 0.036974191665649414 time for calcul the mask position with numpy : 0.027962207794189453 nb_pixel_total : 189042 time to create 1 rle with new method : 0.6522290706634521 time for calcul the mask position with numpy : 0.0240480899810791 nb_pixel_total : 12272 time to create 1 rle with old method : 0.012346267700195312 time for calcul the mask position with numpy : 0.023950576782226562 nb_pixel_total : 31148 time to create 1 rle with old method : 0.03171896934509277 time for calcul the mask position with numpy : 0.025084257125854492 nb_pixel_total : 12559 time to create 1 rle with old method : 0.014319181442260742 create new chi : 3.2725377082824707 time to delete rle : 0.0016834735870361328 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 9971 TO DO : save crop sub photo not yet done ! save time : 0.8522441387176514 nb_obj : 20 nb_hashtags : 6 time to prepare the origin masks : 14.029694557189941 time for calcul the mask position with numpy : 0.18389201164245605 nb_pixel_total : 4207183 time to create 1 rle with new method : 0.8607058525085449 time for calcul the mask position with numpy : 0.031746625900268555 nb_pixel_total : 1614 time to create 1 rle with old method : 0.0020117759704589844 time for calcul the mask position with numpy : 0.0373837947845459 nb_pixel_total : 7230 time to create 1 rle with old method : 0.007302045822143555 time for calcul the mask position with numpy : 0.026079416275024414 nb_pixel_total : 1005 time to create 1 rle with old method : 0.0012369155883789062 time for calcul the mask position with numpy : 0.03048872947692871 nb_pixel_total : 51222 time to create 1 rle with old method : 0.05360293388366699 time for calcul the mask position with numpy : 0.030112266540527344 nb_pixel_total : 135657 time to create 1 rle with old method : 0.13486480712890625 time for calcul the mask position with numpy : 0.027253389358520508 nb_pixel_total : 82767 time to create 1 rle with old method : 0.08580636978149414 time for calcul the mask position with numpy : 0.024274349212646484 nb_pixel_total : 2629 time to create 1 rle with old method : 0.0032660961151123047 time for calcul the mask position with numpy : 0.031150102615356445 nb_pixel_total : 511995 time to create 1 rle with new method : 0.47076869010925293 time for calcul the mask position with numpy : 0.0268099308013916 nb_pixel_total : 2998 time to create 1 rle with old method : 0.0031900405883789062 time for calcul the mask position with numpy : 0.024662494659423828 nb_pixel_total : 3835 time to create 1 rle with old method : 0.00412750244140625 time for calcul the mask position with numpy : 0.027499675750732422 nb_pixel_total : 53565 time to create 1 rle with old method : 0.05484628677368164 time for calcul the mask position with numpy : 0.025176525115966797 nb_pixel_total : 4637 time to create 1 rle with old method : 0.005242347717285156 time for calcul the mask position with numpy : 0.026125192642211914 nb_pixel_total : 117692 time to create 1 rle with old method : 0.12216067314147949 time for calcul the mask position with numpy : 0.027083158493041992 nb_pixel_total : 90426 time to create 1 rle with old method : 0.09319782257080078 time for calcul the mask position with numpy : 0.0265657901763916 nb_pixel_total : 2220 time to create 1 rle with old method : 0.002489328384399414 time for calcul the mask position with numpy : 0.027100086212158203 nb_pixel_total : 152605 time to create 1 rle with new method : 0.5598328113555908 time for calcul the mask position with numpy : 0.02746272087097168 nb_pixel_total : 6155 time to create 1 rle with old method : 0.006863832473754883 time for calcul the mask position with numpy : 0.04737043380737305 nb_pixel_total : 1498473 time to create 1 rle with new method : 0.8119795322418213 time for calcul the mask position with numpy : 0.0323026180267334 nb_pixel_total : 1656 time to create 1 rle with old method : 0.0020661354064941406 time for calcul the mask position with numpy : 0.02879166603088379 nb_pixel_total : 114676 time to create 1 rle with old method : 0.12554383277893066 create new chi : 4.335669994354248 time to delete rle : 0.0018646717071533203 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 18435 TO DO : save crop sub photo not yet done ! save time : 3.0151774883270264 nb_obj : 17 nb_hashtags : 6 time to prepare the origin masks : 11.98940134048462 time for calcul the mask position with numpy : 0.3740544319152832 nb_pixel_total : 6406118 time to create 1 rle with new method : 0.594825267791748 time for calcul the mask position with numpy : 0.021868228912353516 nb_pixel_total : 46363 time to create 1 rle with old method : 0.04844188690185547 time for calcul the mask position with numpy : 0.023120403289794922 nb_pixel_total : 137242 time to create 1 rle with old method : 0.14610743522644043 time for calcul the mask position with numpy : 0.020597457885742188 nb_pixel_total : 9976 time to create 1 rle with old method : 0.011138439178466797 time for calcul the mask position with numpy : 0.020357847213745117 nb_pixel_total : 11332 time to create 1 rle with old method : 0.012025117874145508 time for calcul the mask position with numpy : 0.021070480346679688 nb_pixel_total : 71490 time to create 1 rle with old method : 0.07415008544921875 time for calcul the mask position with numpy : 0.021485567092895508 nb_pixel_total : 11861 time to create 1 rle with old method : 0.012441635131835938 time for calcul the mask position with numpy : 0.021100997924804688 nb_pixel_total : 8410 time to create 1 rle with old method : 0.008934259414672852 time for calcul the mask position with numpy : 0.0230865478515625 nb_pixel_total : 22178 time to create 1 rle with old method : 0.023844242095947266 time for calcul the mask position with numpy : 0.023442506790161133 nb_pixel_total : 17031 time to create 1 rle with old method : 0.0185244083404541 time for calcul the mask position with numpy : 0.02265334129333496 nb_pixel_total : 9526 time to create 1 rle with old method : 0.010042428970336914 time for calcul the mask position with numpy : 0.021185636520385742 nb_pixel_total : 144082 time to create 1 rle with old method : 0.14897751808166504 time for calcul the mask position with numpy : 0.020464658737182617 nb_pixel_total : 21004 time to create 1 rle with old method : 0.022550106048583984 time for calcul the mask position with numpy : 0.02127861976623535 nb_pixel_total : 53 time to create 1 rle with old method : 0.00021505355834960938 time for calcul the mask position with numpy : 0.02216506004333496 nb_pixel_total : 56264 time to create 1 rle with old method : 0.06381988525390625 time for calcul the mask position with numpy : 0.02175760269165039 nb_pixel_total : 2868 time to create 1 rle with old method : 0.003310680389404297 time for calcul the mask position with numpy : 0.02188277244567871 nb_pixel_total : 54171 time to create 1 rle with old method : 0.05642509460449219 time for calcul the mask position with numpy : 0.0222170352935791 nb_pixel_total : 20271 time to create 1 rle with old method : 0.021839618682861328 create new chi : 2.0643889904022217 time to delete rle : 0.0010895729064941406 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 11145 TO DO : save crop sub photo not yet done ! save time : 1.8757829666137695 nb_obj : 18 nb_hashtags : 6 time to prepare the origin masks : 9.289885759353638 time for calcul the mask position with numpy : 0.4522073268890381 nb_pixel_total : 6017449 time to create 1 rle with new method : 1.8386075496673584 time for calcul the mask position with numpy : 0.02612924575805664 nb_pixel_total : 28532 time to create 1 rle with old method : 0.030579805374145508 time for calcul the mask position with numpy : 0.03439617156982422 nb_pixel_total : 2678 time to create 1 rle with old method : 0.003058195114135742 time for calcul the mask position with numpy : 0.035248756408691406 nb_pixel_total : 59002 time to create 1 rle with old method : 0.05993533134460449 time for calcul the mask position with numpy : 0.038185834884643555 nb_pixel_total : 378187 time to create 1 rle with new method : 1.0046236515045166 time for calcul the mask position with numpy : 0.022382259368896484 nb_pixel_total : 122118 time to create 1 rle with old method : 0.12247586250305176 time for calcul the mask position with numpy : 0.021783113479614258 nb_pixel_total : 5885 time to create 1 rle with old method : 0.006434440612792969 time for calcul the mask position with numpy : 0.022392988204956055 nb_pixel_total : 7862 time to create 1 rle with old method : 0.00878143310546875 time for calcul the mask position with numpy : 0.022440671920776367 nb_pixel_total : 45640 time to create 1 rle with old method : 0.046890974044799805 time for calcul the mask position with numpy : 0.02153635025024414 nb_pixel_total : 3271 time to create 1 rle with old method : 0.003579378128051758 time for calcul the mask position with numpy : 0.022504568099975586 nb_pixel_total : 5929 time to create 1 rle with old method : 0.006288766860961914 time for calcul the mask position with numpy : 0.022492408752441406 nb_pixel_total : 115002 time to create 1 rle with old method : 0.11473441123962402 time for calcul the mask position with numpy : 0.02226543426513672 nb_pixel_total : 1990 time to create 1 rle with old method : 0.002084970474243164 time for calcul the mask position with numpy : 0.020968914031982422 nb_pixel_total : 10406 time to create 1 rle with old method : 0.011479854583740234 time for calcul the mask position with numpy : 0.021619796752929688 nb_pixel_total : 14225 time to create 1 rle with old method : 0.014275550842285156 time for calcul the mask position with numpy : 0.021553993225097656 nb_pixel_total : 5094 time to create 1 rle with old method : 0.005303382873535156 time for calcul the mask position with numpy : 0.023200273513793945 nb_pixel_total : 169157 time to create 1 rle with new method : 0.32652735710144043 time for calcul the mask position with numpy : 0.023654699325561523 nb_pixel_total : 2517 time to create 1 rle with old method : 0.002827167510986328 time for calcul the mask position with numpy : 0.022629737854003906 nb_pixel_total : 55296 time to create 1 rle with old method : 0.06072854995727539 create new chi : 4.648025751113892 time to delete rle : 0.0011286735534667969 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 11322 TO DO : save crop sub photo not yet done ! save time : 2.2052035331726074 nb_obj : 9 nb_hashtags : 4 time to prepare the origin masks : 10.732577800750732 time for calcul the mask position with numpy : 0.3418567180633545 nb_pixel_total : 6752049 time to create 1 rle with new method : 0.541374921798706 time for calcul the mask position with numpy : 0.0246732234954834 nb_pixel_total : 13558 time to create 1 rle with old method : 0.014921903610229492 time for calcul the mask position with numpy : 0.022464752197265625 nb_pixel_total : 1925 time to create 1 rle with old method : 0.002244234085083008 time for calcul the mask position with numpy : 0.02108025550842285 nb_pixel_total : 47700 time to create 1 rle with old method : 0.05145692825317383 time for calcul the mask position with numpy : 0.021099328994750977 nb_pixel_total : 11355 time to create 1 rle with old method : 0.013423681259155273 time for calcul the mask position with numpy : 0.02163100242614746 nb_pixel_total : 70998 time to create 1 rle with old method : 0.07845115661621094 time for calcul the mask position with numpy : 0.021173715591430664 nb_pixel_total : 7615 time to create 1 rle with old method : 0.009277820587158203 time for calcul the mask position with numpy : 0.021007061004638672 nb_pixel_total : 11019 time to create 1 rle with old method : 0.012339115142822266 time for calcul the mask position with numpy : 0.02729034423828125 nb_pixel_total : 26786 time to create 1 rle with old method : 0.03138923645019531 time for calcul the mask position with numpy : 0.02267932891845703 nb_pixel_total : 107235 time to create 1 rle with old method : 0.1385643482208252 create new chi : 1.4928803443908691 time to delete rle : 0.003673076629638672 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 7114 TO DO : save crop sub photo not yet done ! save time : 1.123790979385376 map_output_result : {1336797885: (0.0, 'Should be the crop_list due to order', 0.0), 1336797871: (0.0, 'Should be the crop_list due to order', 0.0), 1336797806: (0.0, 'Should be the crop_list due to order', 0.0), 1336797703: (0.0, 'Should be the crop_list due to order', 0.0), 1336796211: (0.0, 'Should be the crop_list due to order', 0.0), 1336796103: (0.0, 'Should be the crop_list due to order', 0.0), 1336794936: (0.0, 'Should be the crop_list due to order', 0.0), 1336794898: (0.0, 'Should be the crop_list due to order', 0.0), 1336794754: (0.0, 'Should be the crop_list due to order', 0.0), 1336794688: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1336797885, 1336797871, 1336797806, 1336797703, 1336796211, 1336796103, 1336794936, 1336794898, 1336794754, 1336794688] Looping around the photos to save general results len do output : 10 /1336797885.Didn't retrieve data . /1336797871.Didn't retrieve data . /1336797806.Didn't retrieve data . /1336797703.Didn't retrieve data . /1336796211.Didn't retrieve data . /1336796103.Didn't retrieve data . /1336794936.Didn't retrieve data . /1336794898.Didn't retrieve data . /1336794754.Didn't retrieve data . /1336794688.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797885', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797871', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797806', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797703', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796211', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796103', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794936', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794898', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794754', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794688', None, None, None, None, None, '2578112') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.1312110424041748 save_final save missing photos in datou_result : time spend for datou_step_exec : 150.18289732933044 time spend to save output : 0.3556811809539795 total time spend for step 5 : 150.53857851028442 step6:crop_condition Wed Feb 12 01:17:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 177 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739319429_930758 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.9888920783996582 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739319436_930758 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.9228641986846924 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739319442_930758 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.253208875656128 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739319444_930758 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.565971851348877 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739319448_930758 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.592376708984375 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739319453_930758 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.164698839187622 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739319456_930758 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.892730712890625 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1336797885, 1336797871, 1336797806, 1336797703, 1336796211, 1336796103, 1336794936, 1336794898, 1336794754, 1336794688] Looping around the photos to save general results len do output : 30 /1336955881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955882Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955889Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955891Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955892Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955894Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955896Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955900Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955908Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955912Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955921Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955922Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336955927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797885', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797871', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797806', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797703', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796211', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796103', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794936', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794898', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794754', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794688', None, None, None, None, None, '2578112') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 100 time used for this insertion : 0.1634056568145752 save_final save missing photos in datou_result : time spend for datou_step_exec : 35.01652693748474 time spend to save output : 0.4173119068145752 total time spend for step 6 : 35.433838844299316 step7:ventilate_hashtags_in_portfolio Wed Feb 12 01:17:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20464976 get user id for portfolio 20464976 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464976 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','kraft','plastique','autre_refus','metal','papier','Carton_brun','environnement','flou','cartonnette','Teint_Dans_La_Masse','mal_croppe')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464976 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','kraft','plastique','autre_refus','metal','papier','Carton_brun','environnement','flou','cartonnette','Teint_Dans_La_Masse','mal_croppe')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464976 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','kraft','plastique','autre_refus','metal','papier','Carton_brun','environnement','flou','cartonnette','Teint_Dans_La_Masse','mal_croppe')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20464986,20464987,20464988,20464989,20464990,20464991,20464992,20464993,20464994,20464995,20464996,20464997?tags=Carton_gris,kraft,plastique,autre_refus,metal,papier,Carton_brun,environnement,flou,cartonnette,Teint_Dans_La_Masse,mal_croppe Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1336797885, 1336797871, 1336797806, 1336797703, 1336796211, 1336796103, 1336794936, 1336794898, 1336794754, 1336794688] Looping around the photos to save general results len do output : 1 /20464976. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797885', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797871', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797806', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797703', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796211', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796103', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794936', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794898', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794754', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794688', None, None, None, None, None, '2578112') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.016078948974609375 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.942716360092163 time spend to save output : 0.01634073257446289 total time spend for step 7 : 1.959057092666626 step8:final Wed Feb 12 01:17:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1336797885: ('0.0877803823836981',), 1336797871: ('0.0877803823836981',), 1336797806: ('0.0877803823836981',), 1336797703: ('0.0877803823836981',), 1336796211: ('0.0877803823836981',), 1336796103: ('0.0877803823836981',), 1336794936: ('0.0877803823836981',), 1336794898: ('0.0877803823836981',), 1336794754: ('0.0877803823836981',), 1336794688: ('0.0877803823836981',)} new output for save of step final : {1336797885: ('0.0877803823836981',), 1336797871: ('0.0877803823836981',), 1336797806: ('0.0877803823836981',), 1336797703: ('0.0877803823836981',), 1336796211: ('0.0877803823836981',), 1336796103: ('0.0877803823836981',), 1336794936: ('0.0877803823836981',), 1336794898: ('0.0877803823836981',), 1336794754: ('0.0877803823836981',), 1336794688: ('0.0877803823836981',)} [1336797885, 1336797871, 1336797806, 1336797703, 1336796211, 1336796103, 1336794936, 1336794898, 1336794754, 1336794688] Looping around the photos to save general results len do output : 10 /1336797885.Didn't retrieve data . /1336797871.Didn't retrieve data . /1336797806.Didn't retrieve data . /1336797703.Didn't retrieve data . /1336796211.Didn't retrieve data . /1336796103.Didn't retrieve data . /1336794936.Didn't retrieve data . /1336794898.Didn't retrieve data . /1336794754.Didn't retrieve data . /1336794688.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797885', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797871', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797806', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797703', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796211', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796103', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794936', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794898', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794754', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794688', None, None, None, None, None, '2578112') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.016356229782104492 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10810065269470215 time spend to save output : 0.017025232315063477 total time spend for step 8 : 0.12512588500976562 step9:velours_tree Wed Feb 12 01:17:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.342745304107666 time spend to save output : 6.532669067382812e-05 total time spend for step 9 : 0.34281063079833984 step10:send_mail_cod Wed Feb 12 01:17:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20464976_12-02-2025_01_17_43.pdf 20464987 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204649871739319463 20464988 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204649881739319464 20464989 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204649891739319464 20464990 imagette204649901739319465 20464994 imagette204649941739319465 20464995 change filename to text .change filename to text .change filename to text .change filename to text .imagette204649951739319465 20464997 imagette204649971739319465 20464986 change filename to text .change filename to text .change filename to text .change filename to text .imagette204649861739319465 20464992 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204649921739319466 20464996 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204649961739319466 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20464976 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20464986,20464987,20464988,20464989,20464990,20464991,20464992,20464993,20464994,20464995,20464996,20464997?tags=Carton_gris,kraft,plastique,autre_refus,metal,papier,Carton_brun,environnement,flou,cartonnette,Teint_Dans_La_Masse,mal_croppe your option no_mail is active, we will not send the real mail to your client args[1336797885] : ('0.0877803823836981',) no score found for photo 1336797885 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336797871] : ('0.0877803823836981',) no score found for photo 1336797871 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336797806] : ('0.0877803823836981',) no score found for photo 1336797806 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336797703] : ('0.0877803823836981',) no score found for photo 1336797703 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336796211] : ('0.0877803823836981',) no score found for photo 1336796211 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336796103] : ('0.0877803823836981',) no score found for photo 1336796103 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336794936] : ('0.0877803823836981',) no score found for photo 1336794936 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336794898] : ('0.0877803823836981',) no score found for photo 1336794898 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336794754] : ('0.0877803823836981',) no score found for photo 1336794754 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com args[1336794688] : ('0.0877803823836981',) no score found for photo 1336794688 apple ('0.0877803823836981',) We are sending mail with results at report@fotonower.com refus_total : 0.0877803823836981 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20464976 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1336793226,1336797871,1336796211,1336794754,1336794688,1336793567,1336793492,1336793423,1336793329,1336794898,1336796103,1336794462,1336794936,1336797703,1336797806,1336793112,1336797885) Found this number of photos: 17 begin to download photo : 1336793226 begin to download photo : 1336794688 begin to download photo : 1336793329 begin to download photo : 1336794936 begin to download photo : 1336797885 download finish for photo 1336794688 begin to download photo : 1336793567 download finish for photo 1336793329 begin to download photo : 1336794898 download finish for photo 1336794936 begin to download photo : 1336797703 download finish for photo 1336793226 begin to download photo : 1336797871 download finish for photo 1336797885 download finish for photo 1336797871 begin to download photo : 1336796211 download finish for photo 1336794898 begin to download photo : 1336796103 download finish for photo 1336797703 begin to download photo : 1336797806 download finish for photo 1336793567 begin to download photo : 1336793492 download finish for photo 1336796211 begin to download photo : 1336794754 download finish for photo 1336796103 begin to download photo : 1336794462 download finish for photo 1336793492 begin to download photo : 1336793423 download finish for photo 1336797806 begin to download photo : 1336793112 download finish for photo 1336794462 download finish for photo 1336794754 download finish for photo 1336793423 download finish for photo 1336793112 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464976_12-02-2025_01_17_43.pdf results_Auto_P20464976_12-02-2025_01_17_43.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464976_12-02-2025_01_17_43.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20464976','results_Auto_P20464976_12-02-2025_01_17_43.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464976_12-02-2025_01_17_43.pdf','pdf','','0.93','0.0877803823836981') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1336797885, 1336797871, 1336797806, 1336797703, 1336796211, 1336796103, 1336794936, 1336794898, 1336794754, 1336794688] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797885', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797871', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797806', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797703', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796211', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796103', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794936', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794898', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794754', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794688', None, None, None, None, None, '2578112') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.2346179485321045 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.587036848068237 time spend to save output : 0.23549342155456543 total time spend for step 10 : 7.822530269622803 step11:split_time_score Wed Feb 12 01:17:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('11', 17),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11022025 20464976 Nombre de photos uploadées : 17 / 23040 (0%) 11022025 20464976 Nombre de photos taguées (types de déchets): 0 / 17 (0%) 11022025 20464976 Nombre de photos taguées (volume) : 0 / 17 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 1.2874603271484375e-05 ????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.00395965576171875 elapsed_time : insert_dashboard_record_day_entry 0.02407073974609375 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.12175292317109736 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464066_12-02-2025_01_12_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464066 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464066 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464970 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464971 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464972 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464973 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464974 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461693 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461694 order by id desc limit 1 Qualite : 0.04664011771303797 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20461695_12-02-2025_00_07_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461695 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20461695 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464975 order by id desc limit 1 Qualite : 0.0877803823836981 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464976_12-02-2025_01_17_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464976 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464976 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464977 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464978 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464979 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464980 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464981 order by id desc limit 1 Qualite : 0.08092846174692254 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464068_12-02-2025_01_15_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464068 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464068 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464070 order by id desc limit 1 Qualite : 0.0698621599574667 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464072_12-02-2025_00_39_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464072 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464072 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464073 order by id desc limit 1 Qualite : 0.14894435070972942 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464075_12-02-2025_01_08_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464075 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464075 AND mptpi.`type`=3327 To do Qualite : 0.07290612071760248 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464077_12-02-2025_01_00_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464077 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464077 AND mptpi.`type`=4211 To do Qualite : 0.44956682089365013 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464079_12-02-2025_01_15_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464079 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464079 AND mptpi.`type`=4200 To do Qualite : 0.0847244111430594 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464081_12-02-2025_01_11_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464081 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464081 AND mptpi.`type`=4207 To do Qualite : 0.055549930801444956 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464083_12-02-2025_00_47_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464083 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464083 AND mptpi.`type`=4207 To do Qualite : 0.06649324540749325 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20464085_12-02-2025_00_34_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20464085 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20464085 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462397 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462398 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462399 order by id desc limit 1 Qualite : 0.2073358909294812 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462400_11-02-2025_22_44_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462400 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462400 AND mptpi.`type`=3594 To do Qualite : 0.35079116052729603 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462401_11-02-2025_22_34_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462401 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462401 AND mptpi.`type`=4200 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462402 order by id desc limit 1 Qualite : 0.21968674510056066 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20461710_11-02-2025_22_48_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461710 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20461710 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461711 order by id desc limit 1 Qualite : 0.11839248806077622 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20461712_11-02-2025_21_36_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461712 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20461712 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461713 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461714 order by id desc limit 1 Qualite : 0.03686876657891321 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462403_11-02-2025_23_11_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462403 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462403 AND mptpi.`type`=4207 To do Qualite : 0.042788788359534105 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462404_11-02-2025_23_04_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462404 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462404 AND mptpi.`type`=4207 To do Qualite : 0.09319486151975546 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462405_11-02-2025_22_50_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462405 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462405 AND mptpi.`type`=3327 To do Qualite : 0.12503171394294246 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20462406_11-02-2025_23_21_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20462406 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20462406 AND mptpi.`type`=3594 To do Qualite : 0.11399216960644655 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20461716_11-02-2025_23_37_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461716 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20461716 AND mptpi.`type`=3594 To do Qualite : 0.09091100444807554 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20461717_11-02-2025_21_35_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461717 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20461717 AND mptpi.`type`=3594 To do Qualite : 0.02953539912490668 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20461718_11-02-2025_21_36_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461718 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20461718 AND mptpi.`type`=4207 To do Qualite : 0.041030917312523105 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20461719_11-02-2025_22_16_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20461719 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20461719 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11022025': {'nb_upload': 17, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1336797885, 1336797871, 1336797806, 1336797703, 1336796211, 1336796103, 1336794936, 1336794898, 1336794754, 1336794688] Looping around the photos to save general results len do output : 1 /20464976Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797885', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797871', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797806', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336797703', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796211', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336796103', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794936', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794898', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794754', None, None, None, None, None, '2578112') ('3459', None, None, None, None, None, None, None, '2578112') ('3459', '20464976', '1336794688', None, None, None, None, None, '2578112') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.0193021297454834 save_final save missing photos in datou_result : time spend for datou_step_exec : 18.964994430541992 time spend to save output : 0.019812822341918945 total time spend for step 11 : 18.98480725288391 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 151.26user 141.73system 5:47.52elapsed 84%CPU (0avgtext+0avgdata 4421892maxresident)k 1594792inputs+141008outputs (19764major+12434138minor)pagefaults 0swaps