python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1309421 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 24>10 please execute split_portfolio.py -i 21290053 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2658419'] with mtr_portfolio_ids : ['21290053'] and first list_photo_ids : [] new path : /proc/1309421/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.7650437355041504 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Mar 11 14:56:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-11 14:56:32.887377: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-11 14:56:32.915060: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-11 14:56:32.917226: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f15c4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-11 14:56:32.917294: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-11 14:56:32.921549: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-11 14:56:33.169949: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2b419600 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-11 14:56:33.169992: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-11 14:56:33.170877: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 14:56:33.171208: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 14:56:33.173299: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 14:56:33.175799: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 14:56:33.176156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 14:56:33.179029: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 14:56:33.180177: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 14:56:33.185029: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 14:56:33.186483: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 14:56:33.186567: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 14:56:33.187355: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 14:56:33.187373: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 14:56:33.187382: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 14:56:33.188664: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9607 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-11 14:56:33.460983: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 14:56:33.461071: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 14:56:33.461092: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 14:56:33.461111: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 14:56:33.461129: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 14:56:33.461147: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 14:56:33.461165: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 14:56:33.461182: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 14:56:33.462729: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 14:56:33.464122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 14:56:33.464167: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 14:56:33.464185: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 14:56:33.464202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 14:56:33.464219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 14:56:33.464236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 14:56:33.464253: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 14:56:33.464270: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 14:56:33.465828: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 14:56:33.465859: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 14:56:33.465870: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 14:56:33.465880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 14:56:33.467528: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9607 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-11 14:56:43.874185: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 14:56:44.051690: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 34 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.63302 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 34 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 34 Detection mask done ! Trying to reset tf kernel 1310071 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5083 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10374 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0004744529724121094 nb_pixel_total : 15267 time to create 1 rle with old method : 0.018496990203857422 length of segment : 138 time for calcul the mask position with numpy : 2.2548961639404297 nb_pixel_total : 3760087 time to create 1 rle with new method : 0.6344528198242188 length of segment : 3557 time for calcul the mask position with numpy : 0.0011916160583496094 nb_pixel_total : 91875 time to create 1 rle with old method : 0.10461211204528809 length of segment : 360 time for calcul the mask position with numpy : 0.0003848075866699219 nb_pixel_total : 8844 time to create 1 rle with old method : 0.011777639389038086 length of segment : 253 time for calcul the mask position with numpy : 0.00030684471130371094 nb_pixel_total : 14163 time to create 1 rle with old method : 0.01775360107421875 length of segment : 120 time for calcul the mask position with numpy : 0.0019199848175048828 nb_pixel_total : 169520 time to create 1 rle with new method : 0.004767179489135742 length of segment : 354 time for calcul the mask position with numpy : 0.0006470680236816406 nb_pixel_total : 38706 time to create 1 rle with old method : 0.05079388618469238 length of segment : 415 time for calcul the mask position with numpy : 0.0001270771026611328 nb_pixel_total : 2916 time to create 1 rle with old method : 0.005078792572021484 length of segment : 63 time for calcul the mask position with numpy : 0.00011134147644042969 nb_pixel_total : 3515 time to create 1 rle with old method : 0.006140470504760742 length of segment : 44 time for calcul the mask position with numpy : 0.003139972686767578 nb_pixel_total : 169611 time to create 1 rle with new method : 0.008018970489501953 length of segment : 451 time for calcul the mask position with numpy : 0.0005204677581787109 nb_pixel_total : 16919 time to create 1 rle with old method : 0.023894071578979492 length of segment : 147 time for calcul the mask position with numpy : 0.0018813610076904297 nb_pixel_total : 67841 time to create 1 rle with old method : 0.08665657043457031 length of segment : 454 time for calcul the mask position with numpy : 0.0018720626831054688 nb_pixel_total : 94752 time to create 1 rle with old method : 0.11241769790649414 length of segment : 294 time for calcul the mask position with numpy : 0.00028252601623535156 nb_pixel_total : 11997 time to create 1 rle with old method : 0.015066146850585938 length of segment : 148 time for calcul the mask position with numpy : 7.963180541992188e-05 nb_pixel_total : 1791 time to create 1 rle with old method : 0.002290010452270508 length of segment : 60 time for calcul the mask position with numpy : 0.0012440681457519531 nb_pixel_total : 46433 time to create 1 rle with old method : 0.054952383041381836 length of segment : 362 time for calcul the mask position with numpy : 0.0010461807250976562 nb_pixel_total : 50827 time to create 1 rle with old method : 0.06263971328735352 length of segment : 416 time for calcul the mask position with numpy : 8.225440979003906e-05 nb_pixel_total : 1779 time to create 1 rle with old method : 0.0021479129791259766 length of segment : 54 time for calcul the mask position with numpy : 0.002171754837036133 nb_pixel_total : 151965 time to create 1 rle with new method : 0.005143404006958008 length of segment : 408 time for calcul the mask position with numpy : 0.0008842945098876953 nb_pixel_total : 38410 time to create 1 rle with old method : 0.04742574691772461 length of segment : 266 time for calcul the mask position with numpy : 0.0005333423614501953 nb_pixel_total : 20932 time to create 1 rle with old method : 0.02511429786682129 length of segment : 178 time for calcul the mask position with numpy : 0.0008885860443115234 nb_pixel_total : 47817 time to create 1 rle with old method : 0.061385393142700195 length of segment : 149 time for calcul the mask position with numpy : 0.0009496212005615234 nb_pixel_total : 50177 time to create 1 rle with old method : 0.0603792667388916 length of segment : 296 time for calcul the mask position with numpy : 0.0008268356323242188 nb_pixel_total : 34749 time to create 1 rle with old method : 0.047501564025878906 length of segment : 228 time for calcul the mask position with numpy : 0.003026723861694336 nb_pixel_total : 129782 time to create 1 rle with old method : 0.15226078033447266 length of segment : 671 time for calcul the mask position with numpy : 0.0024902820587158203 nb_pixel_total : 40785 time to create 1 rle with old method : 0.0538792610168457 length of segment : 665 time for calcul the mask position with numpy : 0.00016832351684570312 nb_pixel_total : 4698 time to create 1 rle with old method : 0.005930185317993164 length of segment : 99 time for calcul the mask position with numpy : 0.0009913444519042969 nb_pixel_total : 52979 time to create 1 rle with old method : 0.06299042701721191 length of segment : 334 time for calcul the mask position with numpy : 0.0007021427154541016 nb_pixel_total : 33537 time to create 1 rle with old method : 0.04122042655944824 length of segment : 206 time for calcul the mask position with numpy : 0.0004956722259521484 nb_pixel_total : 28749 time to create 1 rle with old method : 0.03517556190490723 length of segment : 224 time for calcul the mask position with numpy : 0.0011713504791259766 nb_pixel_total : 43211 time to create 1 rle with old method : 0.055443525314331055 length of segment : 149 time for calcul the mask position with numpy : 0.0007212162017822266 nb_pixel_total : 12629 time to create 1 rle with old method : 0.01518392562866211 length of segment : 175 time for calcul the mask position with numpy : 0.0011594295501708984 nb_pixel_total : 26196 time to create 1 rle with old method : 0.030735254287719727 length of segment : 261 time for calcul the mask position with numpy : 0.0010523796081542969 nb_pixel_total : 30634 time to create 1 rle with old method : 0.03662919998168945 length of segment : 247 time for calcul the mask position with numpy : 0.0011255741119384766 nb_pixel_total : 25148 time to create 1 rle with old method : 0.029541730880737305 length of segment : 212 time for calcul the mask position with numpy : 0.0007233619689941406 nb_pixel_total : 17085 time to create 1 rle with old method : 0.019634008407592773 length of segment : 147 time for calcul the mask position with numpy : 0.0038330554962158203 nb_pixel_total : 102563 time to create 1 rle with old method : 0.11936116218566895 length of segment : 287 time for calcul the mask position with numpy : 0.0009534358978271484 nb_pixel_total : 19352 time to create 1 rle with old method : 0.023375511169433594 length of segment : 189 time for calcul the mask position with numpy : 0.0009970664978027344 nb_pixel_total : 28449 time to create 1 rle with old method : 0.03715038299560547 length of segment : 250 time for calcul the mask position with numpy : 0.004177093505859375 nb_pixel_total : 92776 time to create 1 rle with old method : 0.10432839393615723 length of segment : 655 time for calcul the mask position with numpy : 0.0010869503021240234 nb_pixel_total : 18955 time to create 1 rle with old method : 0.022420406341552734 length of segment : 182 time for calcul the mask position with numpy : 0.0025992393493652344 nb_pixel_total : 103545 time to create 1 rle with old method : 0.11915135383605957 length of segment : 560 time for calcul the mask position with numpy : 0.0032672882080078125 nb_pixel_total : 116295 time to create 1 rle with old method : 0.1492319107055664 length of segment : 593 time for calcul the mask position with numpy : 0.0004303455352783203 nb_pixel_total : 5119 time to create 1 rle with old method : 0.006424427032470703 length of segment : 133 time for calcul the mask position with numpy : 0.0017132759094238281 nb_pixel_total : 42017 time to create 1 rle with old method : 0.051940202713012695 length of segment : 336 time for calcul the mask position with numpy : 0.0018115043640136719 nb_pixel_total : 67917 time to create 1 rle with old method : 0.08081269264221191 length of segment : 285 time for calcul the mask position with numpy : 0.0005116462707519531 nb_pixel_total : 14086 time to create 1 rle with old method : 0.0169980525970459 length of segment : 149 time for calcul the mask position with numpy : 0.0010132789611816406 nb_pixel_total : 50353 time to create 1 rle with old method : 0.05857229232788086 length of segment : 233 time for calcul the mask position with numpy : 0.0016889572143554688 nb_pixel_total : 61878 time to create 1 rle with old method : 0.07226705551147461 length of segment : 315 time for calcul the mask position with numpy : 0.0010237693786621094 nb_pixel_total : 30395 time to create 1 rle with old method : 0.03658437728881836 length of segment : 241 time for calcul the mask position with numpy : 0.00021386146545410156 nb_pixel_total : 4554 time to create 1 rle with old method : 0.005872964859008789 length of segment : 80 time for calcul the mask position with numpy : 0.00022029876708984375 nb_pixel_total : 3797 time to create 1 rle with old method : 0.004640340805053711 length of segment : 103 time for calcul the mask position with numpy : 0.00023674964904785156 nb_pixel_total : 5054 time to create 1 rle with old method : 0.0067484378814697266 length of segment : 61 time for calcul the mask position with numpy : 0.0005710124969482422 nb_pixel_total : 15491 time to create 1 rle with old method : 0.019443988800048828 length of segment : 297 time for calcul the mask position with numpy : 0.0005881786346435547 nb_pixel_total : 47667 time to create 1 rle with old method : 0.05505228042602539 length of segment : 236 time for calcul the mask position with numpy : 0.0009949207305908203 nb_pixel_total : 26620 time to create 1 rle with old method : 0.03255176544189453 length of segment : 338 time for calcul the mask position with numpy : 0.0001506805419921875 nb_pixel_total : 1753 time to create 1 rle with old method : 0.0022432804107666016 length of segment : 68 time for calcul the mask position with numpy : 0.0014796257019042969 nb_pixel_total : 59444 time to create 1 rle with old method : 0.07393813133239746 length of segment : 313 time for calcul the mask position with numpy : 0.0019040107727050781 nb_pixel_total : 79214 time to create 1 rle with old method : 0.11558914184570312 length of segment : 272 time for calcul the mask position with numpy : 0.010727405548095703 nb_pixel_total : 380761 time to create 1 rle with new method : 0.020804405212402344 length of segment : 771 time for calcul the mask position with numpy : 0.0016870498657226562 nb_pixel_total : 52339 time to create 1 rle with old method : 0.060067176818847656 length of segment : 458 time for calcul the mask position with numpy : 0.0003085136413574219 nb_pixel_total : 6414 time to create 1 rle with old method : 0.00791168212890625 length of segment : 110 time for calcul the mask position with numpy : 0.004094600677490234 nb_pixel_total : 184486 time to create 1 rle with new method : 0.005896806716918945 length of segment : 452 time for calcul the mask position with numpy : 0.0011148452758789062 nb_pixel_total : 51807 time to create 1 rle with old method : 0.08157467842102051 length of segment : 305 time for calcul the mask position with numpy : 0.0006906986236572266 nb_pixel_total : 20030 time to create 1 rle with old method : 0.024983882904052734 length of segment : 209 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 1762 time to create 1 rle with old method : 0.002231121063232422 length of segment : 60 time for calcul the mask position with numpy : 0.0010666847229003906 nb_pixel_total : 62117 time to create 1 rle with old method : 0.0732583999633789 length of segment : 303 time for calcul the mask position with numpy : 0.000904083251953125 nb_pixel_total : 8494 time to create 1 rle with old method : 0.010830163955688477 length of segment : 216 time for calcul the mask position with numpy : 0.0009503364562988281 nb_pixel_total : 63408 time to create 1 rle with old method : 0.07581734657287598 length of segment : 304 time for calcul the mask position with numpy : 0.0002956390380859375 nb_pixel_total : 5071 time to create 1 rle with old method : 0.006493806838989258 length of segment : 93 time for calcul the mask position with numpy : 0.004108905792236328 nb_pixel_total : 197969 time to create 1 rle with new method : 0.006194353103637695 length of segment : 460 time for calcul the mask position with numpy : 0.0002532005310058594 nb_pixel_total : 6353 time to create 1 rle with old method : 0.007878303527832031 length of segment : 109 time for calcul the mask position with numpy : 0.002197265625 nb_pixel_total : 58192 time to create 1 rle with old method : 0.06753849983215332 length of segment : 391 time for calcul the mask position with numpy : 0.0006496906280517578 nb_pixel_total : 24545 time to create 1 rle with old method : 0.029618024826049805 length of segment : 143 time for calcul the mask position with numpy : 0.0006086826324462891 nb_pixel_total : 13760 time to create 1 rle with old method : 0.016119718551635742 length of segment : 188 time for calcul the mask position with numpy : 0.0009849071502685547 nb_pixel_total : 43657 time to create 1 rle with old method : 0.05227994918823242 length of segment : 326 time for calcul the mask position with numpy : 0.00044226646423339844 nb_pixel_total : 14275 time to create 1 rle with old method : 0.01818060874938965 length of segment : 160 time for calcul the mask position with numpy : 0.0009980201721191406 nb_pixel_total : 34535 time to create 1 rle with old method : 0.04178810119628906 length of segment : 214 time for calcul the mask position with numpy : 0.003905773162841797 nb_pixel_total : 173313 time to create 1 rle with new method : 0.005835056304931641 length of segment : 748 time for calcul the mask position with numpy : 0.003637075424194336 nb_pixel_total : 132440 time to create 1 rle with old method : 0.15182018280029297 length of segment : 618 time for calcul the mask position with numpy : 0.0008676052093505859 nb_pixel_total : 43157 time to create 1 rle with old method : 0.05117535591125488 length of segment : 404 time for calcul the mask position with numpy : 0.00160980224609375 nb_pixel_total : 153 time to create 1 rle with old method : 0.000274658203125 length of segment : 42 time for calcul the mask position with numpy : 0.005121946334838867 nb_pixel_total : 219386 time to create 1 rle with new method : 0.008150815963745117 length of segment : 900 time for calcul the mask position with numpy : 0.0039055347442626953 nb_pixel_total : 214890 time to create 1 rle with new method : 0.00703120231628418 length of segment : 587 time for calcul the mask position with numpy : 0.0005097389221191406 nb_pixel_total : 11438 time to create 1 rle with old method : 0.014424324035644531 length of segment : 116 time for calcul the mask position with numpy : 0.004427433013916016 nb_pixel_total : 84214 time to create 1 rle with old method : 0.0985414981842041 length of segment : 687 time for calcul the mask position with numpy : 0.0002117156982421875 nb_pixel_total : 2075 time to create 1 rle with old method : 0.002844095230102539 length of segment : 61 time for calcul the mask position with numpy : 0.006216287612915039 nb_pixel_total : 199952 time to create 1 rle with new method : 0.009076118469238281 length of segment : 645 time for calcul the mask position with numpy : 0.0006825923919677734 nb_pixel_total : 9828 time to create 1 rle with old method : 0.011564493179321289 length of segment : 212 time for calcul the mask position with numpy : 0.001191854476928711 nb_pixel_total : 9786 time to create 1 rle with old method : 0.012740373611450195 length of segment : 204 time for calcul the mask position with numpy : 0.004239320755004883 nb_pixel_total : 87761 time to create 1 rle with old method : 0.10793066024780273 length of segment : 413 time for calcul the mask position with numpy : 0.0004546642303466797 nb_pixel_total : 10397 time to create 1 rle with old method : 0.012522697448730469 length of segment : 118 time for calcul the mask position with numpy : 0.0003521442413330078 nb_pixel_total : 7911 time to create 1 rle with old method : 0.009948015213012695 length of segment : 77 time for calcul the mask position with numpy : 0.0002512931823730469 nb_pixel_total : 2039 time to create 1 rle with old method : 0.0028007030487060547 length of segment : 66 time for calcul the mask position with numpy : 0.0012869834899902344 nb_pixel_total : 52901 time to create 1 rle with old method : 0.0622715950012207 length of segment : 414 time for calcul the mask position with numpy : 0.0006930828094482422 nb_pixel_total : 14900 time to create 1 rle with old method : 0.01774883270263672 length of segment : 180 time for calcul the mask position with numpy : 8.177757263183594e-05 nb_pixel_total : 1877 time to create 1 rle with old method : 0.0024700164794921875 length of segment : 74 time for calcul the mask position with numpy : 0.003431081771850586 nb_pixel_total : 87781 time to create 1 rle with old method : 0.10398626327514648 length of segment : 400 time for calcul the mask position with numpy : 0.0008711814880371094 nb_pixel_total : 15102 time to create 1 rle with old method : 0.019945383071899414 length of segment : 89 time for calcul the mask position with numpy : 0.00024700164794921875 nb_pixel_total : 2818 time to create 1 rle with old method : 0.0035636425018310547 length of segment : 73 time for calcul the mask position with numpy : 0.0029268264770507812 nb_pixel_total : 93143 time to create 1 rle with old method : 0.11027383804321289 length of segment : 368 time for calcul the mask position with numpy : 0.001672506332397461 nb_pixel_total : 92391 time to create 1 rle with old method : 0.11477303504943848 length of segment : 368 time for calcul the mask position with numpy : 0.0015022754669189453 nb_pixel_total : 28548 time to create 1 rle with old method : 0.040857553482055664 length of segment : 212 time for calcul the mask position with numpy : 0.002492666244506836 nb_pixel_total : 95334 time to create 1 rle with old method : 0.11353659629821777 length of segment : 405 time for calcul the mask position with numpy : 0.00024628639221191406 nb_pixel_total : 7105 time to create 1 rle with old method : 0.008744478225708008 length of segment : 99 time for calcul the mask position with numpy : 0.0025055408477783203 nb_pixel_total : 50613 time to create 1 rle with old method : 0.0619351863861084 length of segment : 405 time for calcul the mask position with numpy : 0.0012252330780029297 nb_pixel_total : 47559 time to create 1 rle with old method : 0.057045698165893555 length of segment : 202 time for calcul the mask position with numpy : 0.00023126602172851562 nb_pixel_total : 6119 time to create 1 rle with old method : 0.008080720901489258 length of segment : 80 time for calcul the mask position with numpy : 0.011756658554077148 nb_pixel_total : 461051 time to create 1 rle with new method : 0.01777482032775879 length of segment : 724 time for calcul the mask position with numpy : 0.002949953079223633 nb_pixel_total : 45122 time to create 1 rle with old method : 0.07255101203918457 length of segment : 435 time for calcul the mask position with numpy : 0.0013713836669921875 nb_pixel_total : 53826 time to create 1 rle with old method : 0.0713658332824707 length of segment : 223 time for calcul the mask position with numpy : 0.0009503364562988281 nb_pixel_total : 53756 time to create 1 rle with old method : 0.06308436393737793 length of segment : 293 time for calcul the mask position with numpy : 0.0038781166076660156 nb_pixel_total : 164985 time to create 1 rle with new method : 0.00504302978515625 length of segment : 540 time for calcul the mask position with numpy : 0.0016150474548339844 nb_pixel_total : 47526 time to create 1 rle with old method : 0.055947065353393555 length of segment : 430 time for calcul the mask position with numpy : 0.0003097057342529297 nb_pixel_total : 6144 time to create 1 rle with old method : 0.007477283477783203 length of segment : 96 time for calcul the mask position with numpy : 0.0008499622344970703 nb_pixel_total : 50941 time to create 1 rle with old method : 0.05861806869506836 length of segment : 207 time for calcul the mask position with numpy : 0.002803802490234375 nb_pixel_total : 123615 time to create 1 rle with old method : 0.16337203979492188 length of segment : 352 time for calcul the mask position with numpy : 0.0009543895721435547 nb_pixel_total : 56131 time to create 1 rle with old method : 0.06670188903808594 length of segment : 308 time for calcul the mask position with numpy : 0.0001933574676513672 nb_pixel_total : 4276 time to create 1 rle with old method : 0.005808115005493164 length of segment : 73 time for calcul the mask position with numpy : 0.0002846717834472656 nb_pixel_total : 10398 time to create 1 rle with old method : 0.013054609298706055 length of segment : 120 time for calcul the mask position with numpy : 0.0027217864990234375 nb_pixel_total : 101644 time to create 1 rle with old method : 0.12094807624816895 length of segment : 435 time for calcul the mask position with numpy : 0.00015854835510253906 nb_pixel_total : 4665 time to create 1 rle with old method : 0.006032705307006836 length of segment : 73 time for calcul the mask position with numpy : 0.0037276744842529297 nb_pixel_total : 100103 time to create 1 rle with old method : 0.11733126640319824 length of segment : 506 time for calcul the mask position with numpy : 0.0007612705230712891 nb_pixel_total : 13517 time to create 1 rle with old method : 0.017339229583740234 length of segment : 206 time for calcul the mask position with numpy : 0.002679109573364258 nb_pixel_total : 68790 time to create 1 rle with old method : 0.0873112678527832 length of segment : 322 time for calcul the mask position with numpy : 0.0009052753448486328 nb_pixel_total : 25168 time to create 1 rle with old method : 0.031598567962646484 length of segment : 185 time for calcul the mask position with numpy : 0.0004467964172363281 nb_pixel_total : 8730 time to create 1 rle with old method : 0.010392904281616211 length of segment : 103 time for calcul the mask position with numpy : 0.0016007423400878906 nb_pixel_total : 46981 time to create 1 rle with old method : 0.05672717094421387 length of segment : 225 time for calcul the mask position with numpy : 0.0011286735534667969 nb_pixel_total : 26506 time to create 1 rle with old method : 0.0311586856842041 length of segment : 340 time for calcul the mask position with numpy : 0.0008587837219238281 nb_pixel_total : 14319 time to create 1 rle with old method : 0.017020702362060547 length of segment : 211 time for calcul the mask position with numpy : 0.007890939712524414 nb_pixel_total : 268400 time to create 1 rle with new method : 0.011624813079833984 length of segment : 691 time for calcul the mask position with numpy : 0.0006463527679443359 nb_pixel_total : 9774 time to create 1 rle with old method : 0.016604900360107422 length of segment : 147 time for calcul the mask position with numpy : 0.0014698505401611328 nb_pixel_total : 31386 time to create 1 rle with old method : 0.0367586612701416 length of segment : 252 time for calcul the mask position with numpy : 0.0003361701965332031 nb_pixel_total : 5593 time to create 1 rle with old method : 0.006926059722900391 length of segment : 104 time for calcul the mask position with numpy : 0.002389669418334961 nb_pixel_total : 176254 time to create 1 rle with new method : 0.005967140197753906 length of segment : 708 time for calcul the mask position with numpy : 0.011203765869140625 nb_pixel_total : 227415 time to create 1 rle with new method : 0.016472578048706055 length of segment : 720 time for calcul the mask position with numpy : 0.0017278194427490234 nb_pixel_total : 109208 time to create 1 rle with old method : 0.13369393348693848 length of segment : 241 time for calcul the mask position with numpy : 0.0004520416259765625 nb_pixel_total : 9214 time to create 1 rle with old method : 0.011642694473266602 length of segment : 131 time for calcul the mask position with numpy : 0.0005736351013183594 nb_pixel_total : 23628 time to create 1 rle with old method : 0.028226852416992188 length of segment : 327 time for calcul the mask position with numpy : 0.00014448165893554688 nb_pixel_total : 3691 time to create 1 rle with old method : 0.0044896602630615234 length of segment : 109 time for calcul the mask position with numpy : 0.001683950424194336 nb_pixel_total : 39363 time to create 1 rle with old method : 0.04832053184509277 length of segment : 313 time for calcul the mask position with numpy : 0.0004684925079345703 nb_pixel_total : 10831 time to create 1 rle with old method : 0.013677597045898438 length of segment : 93 time for calcul the mask position with numpy : 0.0003249645233154297 nb_pixel_total : 13249 time to create 1 rle with old method : 0.01586627960205078 length of segment : 108 time for calcul the mask position with numpy : 0.002047300338745117 nb_pixel_total : 66594 time to create 1 rle with old method : 0.07837057113647461 length of segment : 477 time for calcul the mask position with numpy : 0.0005533695220947266 nb_pixel_total : 10511 time to create 1 rle with old method : 0.012453317642211914 length of segment : 276 time for calcul the mask position with numpy : 0.004069089889526367 nb_pixel_total : 81534 time to create 1 rle with old method : 0.0929110050201416 length of segment : 691 time for calcul the mask position with numpy : 0.00012421607971191406 nb_pixel_total : 907 time to create 1 rle with old method : 0.0011954307556152344 length of segment : 38 time for calcul the mask position with numpy : 0.00037026405334472656 nb_pixel_total : 7442 time to create 1 rle with old method : 0.008972883224487305 length of segment : 143 time for calcul the mask position with numpy : 0.008145332336425781 nb_pixel_total : 149582 time to create 1 rle with old method : 0.17350435256958008 length of segment : 754 time for calcul the mask position with numpy : 0.000713348388671875 nb_pixel_total : 23608 time to create 1 rle with old method : 0.027609825134277344 length of segment : 167 time for calcul the mask position with numpy : 0.00023937225341796875 nb_pixel_total : 2312 time to create 1 rle with old method : 0.0027811527252197266 length of segment : 89 time for calcul the mask position with numpy : 0.00185394287109375 nb_pixel_total : 73783 time to create 1 rle with old method : 0.08689665794372559 length of segment : 338 time for calcul the mask position with numpy : 0.00024199485778808594 nb_pixel_total : 3567 time to create 1 rle with old method : 0.00456547737121582 length of segment : 72 time for calcul the mask position with numpy : 0.0019826889038085938 nb_pixel_total : 49423 time to create 1 rle with old method : 0.05882120132446289 length of segment : 444 time for calcul the mask position with numpy : 0.00152587890625 nb_pixel_total : 37692 time to create 1 rle with old method : 0.04393720626831055 length of segment : 388 time for calcul the mask position with numpy : 0.0008118152618408203 nb_pixel_total : 22340 time to create 1 rle with old method : 0.027188777923583984 length of segment : 193 time for calcul the mask position with numpy : 0.002692699432373047 nb_pixel_total : 75833 time to create 1 rle with old method : 0.08887672424316406 length of segment : 532 time for calcul the mask position with numpy : 0.0015780925750732422 nb_pixel_total : 45786 time to create 1 rle with old method : 0.05218172073364258 length of segment : 333 time for calcul the mask position with numpy : 0.0006127357482910156 nb_pixel_total : 17855 time to create 1 rle with old method : 0.020707368850708008 length of segment : 168 time for calcul the mask position with numpy : 0.00041413307189941406 nb_pixel_total : 11440 time to create 1 rle with old method : 0.019392728805541992 length of segment : 162 time for calcul the mask position with numpy : 0.006208181381225586 nb_pixel_total : 132083 time to create 1 rle with old method : 0.15972661972045898 length of segment : 547 time for calcul the mask position with numpy : 0.0018184185028076172 nb_pixel_total : 53928 time to create 1 rle with old method : 0.06396722793579102 length of segment : 513 time for calcul the mask position with numpy : 0.003795623779296875 nb_pixel_total : 197648 time to create 1 rle with new method : 0.006396770477294922 length of segment : 624 time for calcul the mask position with numpy : 0.001344919204711914 nb_pixel_total : 41077 time to create 1 rle with old method : 0.04826474189758301 length of segment : 441 time for calcul the mask position with numpy : 0.0022797584533691406 nb_pixel_total : 61247 time to create 1 rle with old method : 0.0740499496459961 length of segment : 292 time for calcul the mask position with numpy : 0.0017292499542236328 nb_pixel_total : 37034 time to create 1 rle with old method : 0.046724796295166016 length of segment : 291 time for calcul the mask position with numpy : 0.0014159679412841797 nb_pixel_total : 62263 time to create 1 rle with old method : 0.07405853271484375 length of segment : 597 time for calcul the mask position with numpy : 0.006276845932006836 nb_pixel_total : 107097 time to create 1 rle with old method : 0.14191865921020508 length of segment : 475 time for calcul the mask position with numpy : 0.001882791519165039 nb_pixel_total : 45946 time to create 1 rle with old method : 0.05507993698120117 length of segment : 431 time for calcul the mask position with numpy : 0.0002155303955078125 nb_pixel_total : 1494 time to create 1 rle with old method : 0.0019731521606445312 length of segment : 75 time for calcul the mask position with numpy : 0.00010132789611816406 nb_pixel_total : 816 time to create 1 rle with old method : 0.0011513233184814453 length of segment : 23 time for calcul the mask position with numpy : 0.0010099411010742188 nb_pixel_total : 29161 time to create 1 rle with old method : 0.037459611892700195 length of segment : 247 time for calcul the mask position with numpy : 0.002236604690551758 nb_pixel_total : 26064 time to create 1 rle with old method : 0.03235888481140137 length of segment : 160 time for calcul the mask position with numpy : 0.001064300537109375 nb_pixel_total : 28453 time to create 1 rle with old method : 0.03377890586853027 length of segment : 231 time for calcul the mask position with numpy : 0.00024318695068359375 nb_pixel_total : 3998 time to create 1 rle with old method : 0.0050389766693115234 length of segment : 90 time for calcul the mask position with numpy : 0.0037384033203125 nb_pixel_total : 76247 time to create 1 rle with old method : 0.08916068077087402 length of segment : 297 time for calcul the mask position with numpy : 0.0005815029144287109 nb_pixel_total : 24686 time to create 1 rle with old method : 0.030299663543701172 length of segment : 212 time for calcul the mask position with numpy : 0.0013778209686279297 nb_pixel_total : 55901 time to create 1 rle with old method : 0.07028031349182129 length of segment : 275 time for calcul the mask position with numpy : 0.0003647804260253906 nb_pixel_total : 6176 time to create 1 rle with old method : 0.007777690887451172 length of segment : 112 time for calcul the mask position with numpy : 0.0074005126953125 nb_pixel_total : 165603 time to create 1 rle with new method : 0.010787487030029297 length of segment : 787 time for calcul the mask position with numpy : 0.017125606536865234 nb_pixel_total : 462577 time to create 1 rle with new method : 0.031078100204467773 length of segment : 903 time for calcul the mask position with numpy : 0.002630472183227539 nb_pixel_total : 71730 time to create 1 rle with old method : 0.0840156078338623 length of segment : 353 time for calcul the mask position with numpy : 0.0002944469451904297 nb_pixel_total : 2629 time to create 1 rle with old method : 0.003446817398071289 length of segment : 40 time for calcul the mask position with numpy : 0.0025899410247802734 nb_pixel_total : 52523 time to create 1 rle with old method : 0.06414914131164551 length of segment : 327 time for calcul the mask position with numpy : 0.0004436969757080078 nb_pixel_total : 6321 time to create 1 rle with old method : 0.01089930534362793 length of segment : 100 time for calcul the mask position with numpy : 0.005616903305053711 nb_pixel_total : 129544 time to create 1 rle with old method : 0.14906072616577148 length of segment : 794 time for calcul the mask position with numpy : 0.0009167194366455078 nb_pixel_total : 37905 time to create 1 rle with old method : 0.04384326934814453 length of segment : 213 time for calcul the mask position with numpy : 0.0021834373474121094 nb_pixel_total : 50417 time to create 1 rle with old method : 0.060349225997924805 length of segment : 282 time for calcul the mask position with numpy : 0.002629518508911133 nb_pixel_total : 27946 time to create 1 rle with old method : 0.03342413902282715 length of segment : 577 time for calcul the mask position with numpy : 0.00020766258239746094 nb_pixel_total : 8399 time to create 1 rle with old method : 0.010289669036865234 length of segment : 142 time for calcul the mask position with numpy : 0.0003485679626464844 nb_pixel_total : 8371 time to create 1 rle with old method : 0.01015472412109375 length of segment : 126 time for calcul the mask position with numpy : 0.002646207809448242 nb_pixel_total : 103178 time to create 1 rle with old method : 0.126115083694458 length of segment : 609 time for calcul the mask position with numpy : 0.004297494888305664 nb_pixel_total : 99029 time to create 1 rle with old method : 0.12228894233703613 length of segment : 482 time for calcul the mask position with numpy : 0.00467228889465332 nb_pixel_total : 108507 time to create 1 rle with old method : 0.13245129585266113 length of segment : 572 time for calcul the mask position with numpy : 0.001621246337890625 nb_pixel_total : 50891 time to create 1 rle with old method : 0.0622713565826416 length of segment : 189 time for calcul the mask position with numpy : 0.0012519359588623047 nb_pixel_total : 20998 time to create 1 rle with old method : 0.026300668716430664 length of segment : 338 time for calcul the mask position with numpy : 0.0011250972747802734 nb_pixel_total : 24944 time to create 1 rle with old method : 0.02961277961730957 length of segment : 317 time for calcul the mask position with numpy : 0.00013399124145507812 nb_pixel_total : 1741 time to create 1 rle with old method : 0.0022721290588378906 length of segment : 47 time for calcul the mask position with numpy : 0.0022101402282714844 nb_pixel_total : 48283 time to create 1 rle with old method : 0.05871915817260742 length of segment : 277 time for calcul the mask position with numpy : 0.0010650157928466797 nb_pixel_total : 22727 time to create 1 rle with old method : 0.02730250358581543 length of segment : 186 time for calcul the mask position with numpy : 0.004048347473144531 nb_pixel_total : 90957 time to create 1 rle with old method : 0.10871028900146484 length of segment : 454 time for calcul the mask position with numpy : 0.004002809524536133 nb_pixel_total : 128433 time to create 1 rle with old method : 0.15506958961486816 length of segment : 402 time for calcul the mask position with numpy : 0.001108407974243164 nb_pixel_total : 33109 time to create 1 rle with old method : 0.0398259162902832 length of segment : 221 time for calcul the mask position with numpy : 0.0007302761077880859 nb_pixel_total : 38319 time to create 1 rle with old method : 0.044339656829833984 length of segment : 288 time for calcul the mask position with numpy : 0.0006999969482421875 nb_pixel_total : 25946 time to create 1 rle with old method : 0.0434870719909668 length of segment : 226 time for calcul the mask position with numpy : 0.00045228004455566406 nb_pixel_total : 4014 time to create 1 rle with old method : 0.005237579345703125 length of segment : 93 time for calcul the mask position with numpy : 0.003451824188232422 nb_pixel_total : 140029 time to create 1 rle with old method : 0.1596376895904541 length of segment : 572 time for calcul the mask position with numpy : 0.0018620491027832031 nb_pixel_total : 46236 time to create 1 rle with old method : 0.05396223068237305 length of segment : 304 time spent for convertir_results : 22.33879566192627 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 208 chid ids of type : 3663 Number RLEs to save : 65090 save missing photos in datou_result : time spend for datou_step_exec : 78.43186807632446 time spend to save output : 71.2950050830841 total time spend for step 1 : 149.72687315940857 step2:crop_condition Tue Mar 11 14:58:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 208 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 41 About to insert : list_path_to_insert length 41 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 58 About to insert : list_path_to_insert length 58 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 140 /-3705556289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556296Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556485Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023136', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023075', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022775', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022749', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022747', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022739', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022531', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022217', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022175', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022101', None, None, None, None, None, '2658419') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 430 time used for this insertion : 1.4070062637329102 save_final save missing photos in datou_result : time spend for datou_step_exec : 57.50283908843994 time spend to save output : 1.4116997718811035 total time spend for step 2 : 58.914538860321045 step3:thcl Tue Mar 11 14:59:58 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.010896682739257812 time to convert the images to numpy array : 0.16798686981201172 time to import caffe and check if the image exist : 0.013449668884277344 time to convert the images to numpy array : 0.16824650764465332 time to import caffe and check if the image exist : 0.01340627670288086 time to convert the images to numpy array : 0.16969799995422363 time to import caffe and check if the image exist : 0.013096332550048828 time to convert the images to numpy array : 0.1754152774810791 time to import caffe and check if the image exist : 0.018450021743774414 time to convert the images to numpy array : 0.4910712242126465 time to import caffe and check if the image exist : 0.012007474899291992 time to convert the images to numpy array : 0.5075516700744629 time to import caffe and check if the image exist : 0.012373924255371094 time to convert the images to numpy array : 0.5167343616485596 time to import caffe and check if the image exist : 0.01564955711364746 time to convert the images to numpy array : 0.5184328556060791 time to import caffe and check if the image exist : 0.012704849243164062 time to convert the images to numpy array : 0.527961254119873 time to import caffe and check if the image exist : 0.01663351058959961 time to convert the images to numpy array : 0.5262444019317627 total time to convert the images to numpy array : 0.7251341342926025 list photo_ids error: [] list photo_ids correct : [-3705556340, -3705556331, -3705556336, -3705556334, -3705556337, -3705556353, -3705556490, -3705556475, -3705556326, -3705556311, -3705556320, -3705556314, -3705556324, -3705556319, -3705556372, -3705556382, -3705556393, -3705556398, -3705556390, -3705556422, -3705556423, -3705556411, -3705556410, -3705556426, -3705556421, -3705556425, -3705556428, -3705556416, -3705556309, -3705556300, -3705556361, -3705556363, -3705556377, -3705556442, -3705556482, -3705556323, -3705556329, -3705556335, -3705556378, -3705556473, -3705556451, -3705556454, -3705556358, -3705556387, -3705556419, -3705556464, -3705556453, -3705556458, -3705556488, -3705556480, -3705556292, -3705556318, -3705556321, -3705556303, -3705556327, -3705556315, -3705556441, -3705556438, -3705556445, -3705556443, -3705556444, -3705556450, -3705556472, -3705556486, -3705556468, -3705556470, -3705556478, -3705556476, -3705556477, -3705556322, -3705556440, -3705556448, -3705556447, -3705556493, -3705556491, -3705556483, -3705556479, -3705556474, -3705556485, -3705556466, -3705556287, -3705556291, -3705556294, -3705556316, -3705556312, -3705556325, -3705556310, -3705556332, -3705556341, -3705556347, -3705556357, -3705556344, -3705556354, -3705556349, -3705556355, -3705556364, -3705556366, -3705556370, -3705556404, -3705556399, -3705556435, -3705556430, -3705556417, -3705556427, -3705556409, -3705556412, -3705556434, -3705556415, -3705556432, -3705556459, -3705556462, -3705556455, -3705556369, -3705556365, -3705556367, -3705556381, -3705556384, -3705556388, -3705556374, -3705556371, -3705556402, -3705556392, -3705556400, -3705556396, -3705556397, -3705556403, -3705556289, -3705556308, -3705556299, -3705556296, -3705556298, -3705556302, -3705556306, -3705556328, -3705556346, -3705556345, -3705556359, -3705556360, -3705556362, -3705556375] number of photos to traite : 140 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10374 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.4558069705963135 time used to do the prediction : 0.27484893798828125 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.4416053295135498 time used to do the prediction : 0.23058199882507324 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.0967254638671875e-05 save missing photos in datou_result : time spend for datou_step_exec : 11.004665613174438 time spend to save output : 0.0008618831634521484 total time spend for step 3 : 11.00552749633789 step4:merge_mask_thcl_custom Tue Mar 11 15:00:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 208 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 10 /1343023136Didn't retrieve data .Didn't retrieve data . /1343023075Didn't retrieve data .Didn't retrieve data . /1343022775Didn't retrieve data .Didn't retrieve data . /1343022749Didn't retrieve data .Didn't retrieve data . /1343022747Didn't retrieve data .Didn't retrieve data . /1343022739Didn't retrieve data .Didn't retrieve data . /1343022531Didn't retrieve data .Didn't retrieve data . /1343022217Didn't retrieve data .Didn't retrieve data . /1343022175Didn't retrieve data .Didn't retrieve data . /1343022101Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023136', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023075', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022775', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022749', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022747', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022739', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022531', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022217', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022175', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022101', None, None, None, None, None, '2658419') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.8173866271972656 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.26241111755371094 time spend to save output : 0.8178086280822754 total time spend for step 4 : 1.0802197456359863 step5:rle_unique_nms_with_priority Tue Mar 11 15:00:10 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 8.201197862625122 time for calcul the mask position with numpy : 0.15555810928344727 nb_pixel_total : 2876582 time to create 1 rle with new method : 1.2144372463226318 time for calcul the mask position with numpy : 0.039777278900146484 nb_pixel_total : 2907 time to create 1 rle with old method : 0.004408836364746094 time for calcul the mask position with numpy : 0.0247194766998291 nb_pixel_total : 15218 time to create 1 rle with old method : 0.025916099548339844 time for calcul the mask position with numpy : 0.02399301528930664 nb_pixel_total : 14141 time to create 1 rle with old method : 0.018863677978515625 time for calcul the mask position with numpy : 0.030003786087036133 nb_pixel_total : 38536 time to create 1 rle with old method : 0.055675506591796875 time for calcul the mask position with numpy : 0.04280233383178711 nb_pixel_total : 137402 time to create 1 rle with old method : 0.17800378799438477 time for calcul the mask position with numpy : 0.03991389274597168 nb_pixel_total : 8767 time to create 1 rle with old method : 0.01317739486694336 time for calcul the mask position with numpy : 0.36635375022888184 nb_pixel_total : 3864996 time to create 1 rle with new method : 1.0039126873016357 time for calcul the mask position with numpy : 0.035093069076538086 nb_pixel_total : 91691 time to create 1 rle with old method : 0.1115560531616211 create new chi : 3.4432179927825928 time to delete rle : 0.015467643737792969 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 11026 TO DO : save crop sub photo not yet done ! save time : 5.498291015625 nb_obj : 32 nb_hashtags : 7 time to prepare the origin masks : 13.462225675582886 time for calcul the mask position with numpy : 1.1433939933776855 nb_pixel_total : 5700985 time to create 1 rle with new method : 0.7499654293060303 time for calcul the mask position with numpy : 0.034851789474487305 nb_pixel_total : 151311 time to create 1 rle with new method : 0.5132265090942383 time for calcul the mask position with numpy : 0.030752182006835938 nb_pixel_total : 43034 time to create 1 rle with old method : 0.05159187316894531 time for calcul the mask position with numpy : 0.029831647872924805 nb_pixel_total : 19323 time to create 1 rle with old method : 0.02256178855895996 time for calcul the mask position with numpy : 0.031913042068481445 nb_pixel_total : 33434 time to create 1 rle with old method : 0.0392606258392334 time for calcul the mask position with numpy : 0.03026556968688965 nb_pixel_total : 16826 time to create 1 rle with old method : 0.020287275314331055 time for calcul the mask position with numpy : 0.0301055908203125 nb_pixel_total : 2051 time to create 1 rle with old method : 0.002720355987548828 time for calcul the mask position with numpy : 0.03052973747253418 nb_pixel_total : 28722 time to create 1 rle with old method : 0.03521466255187988 time for calcul the mask position with numpy : 0.03555774688720703 nb_pixel_total : 38253 time to create 1 rle with old method : 0.05580282211303711 time for calcul the mask position with numpy : 0.03000926971435547 nb_pixel_total : 1769 time to create 1 rle with old method : 0.002263307571411133 time for calcul the mask position with numpy : 0.03046393394470215 nb_pixel_total : 1785 time to create 1 rle with old method : 0.0023736953735351562 time for calcul the mask position with numpy : 0.031065702438354492 nb_pixel_total : 47568 time to create 1 rle with old method : 0.07155966758728027 time for calcul the mask position with numpy : 0.04026174545288086 nb_pixel_total : 129474 time to create 1 rle with old method : 0.16411113739013672 time for calcul the mask position with numpy : 0.03397536277770996 nb_pixel_total : 94589 time to create 1 rle with old method : 0.11825752258300781 time for calcul the mask position with numpy : 0.029929399490356445 nb_pixel_total : 45994 time to create 1 rle with old method : 0.055425167083740234 time for calcul the mask position with numpy : 0.03168153762817383 nb_pixel_total : 22885 time to create 1 rle with old method : 0.03102707862854004 time for calcul the mask position with numpy : 0.03279590606689453 nb_pixel_total : 102203 time to create 1 rle with old method : 0.18036556243896484 time for calcul the mask position with numpy : 0.029751062393188477 nb_pixel_total : 28308 time to create 1 rle with old method : 0.032953739166259766 time for calcul the mask position with numpy : 0.030224084854125977 nb_pixel_total : 26039 time to create 1 rle with old method : 0.031145334243774414 time for calcul the mask position with numpy : 0.035286664962768555 nb_pixel_total : 168324 time to create 1 rle with new method : 0.5497767925262451 time for calcul the mask position with numpy : 0.034790754318237305 nb_pixel_total : 67434 time to create 1 rle with old method : 0.09680485725402832 time for calcul the mask position with numpy : 0.030576467514038086 nb_pixel_total : 30906 time to create 1 rle with old method : 0.038666486740112305 time for calcul the mask position with numpy : 0.029920101165771484 nb_pixel_total : 25082 time to create 1 rle with old method : 0.03034234046936035 time for calcul the mask position with numpy : 0.030831575393676758 nb_pixel_total : 18945 time to create 1 rle with old method : 0.02390289306640625 time for calcul the mask position with numpy : 0.030922889709472656 nb_pixel_total : 3634 time to create 1 rle with old method : 0.0045320987701416016 time for calcul the mask position with numpy : 0.02958536148071289 nb_pixel_total : 11985 time to create 1 rle with old method : 0.014774799346923828 time for calcul the mask position with numpy : 0.03142881393432617 nb_pixel_total : 50068 time to create 1 rle with old method : 0.059567928314208984 time for calcul the mask position with numpy : 0.030786752700805664 nb_pixel_total : 36698 time to create 1 rle with old method : 0.04737114906311035 time for calcul the mask position with numpy : 0.031086206436157227 nb_pixel_total : 12581 time to create 1 rle with old method : 0.01921868324279785 time for calcul the mask position with numpy : 0.03030252456665039 nb_pixel_total : 20731 time to create 1 rle with old method : 0.0240328311920166 time for calcul the mask position with numpy : 0.0305020809173584 nb_pixel_total : 4682 time to create 1 rle with old method : 0.005786418914794922 time for calcul the mask position with numpy : 0.030437469482421875 nb_pixel_total : 17026 time to create 1 rle with old method : 0.028185606002807617 time for calcul the mask position with numpy : 0.03388214111328125 nb_pixel_total : 47591 time to create 1 rle with old method : 0.056731224060058594 create new chi : 5.4280149936676025 time to delete rle : 0.0023534297943115234 batch 1 Loaded 33 chid ids of type : 3726 Number RLEs to save : 18129 TO DO : save crop sub photo not yet done ! save time : 7.4767746925354 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 10.816438674926758 time for calcul the mask position with numpy : 0.23403024673461914 nb_pixel_total : 6540937 time to create 1 rle with new method : 0.5633938312530518 time for calcul the mask position with numpy : 0.03506898880004883 nb_pixel_total : 11565 time to create 1 rle with old method : 0.014175176620483398 time for calcul the mask position with numpy : 0.022797346115112305 nb_pixel_total : 67315 time to create 1 rle with old method : 0.0813589096069336 time for calcul the mask position with numpy : 0.022921085357666016 nb_pixel_total : 30298 time to create 1 rle with old method : 0.03620147705078125 time for calcul the mask position with numpy : 0.024010658264160156 nb_pixel_total : 118430 time to create 1 rle with old method : 0.13712263107299805 time for calcul the mask position with numpy : 0.02835679054260254 nb_pixel_total : 103494 time to create 1 rle with old method : 0.12802720069885254 time for calcul the mask position with numpy : 0.027110576629638672 nb_pixel_total : 41892 time to create 1 rle with old method : 0.0489201545715332 time for calcul the mask position with numpy : 0.0272824764251709 nb_pixel_total : 707 time to create 1 rle with old method : 0.0012264251708984375 time for calcul the mask position with numpy : 0.02856302261352539 nb_pixel_total : 14047 time to create 1 rle with old method : 0.016610383987426758 time for calcul the mask position with numpy : 0.027211904525756836 nb_pixel_total : 4533 time to create 1 rle with old method : 0.005412578582763672 time for calcul the mask position with numpy : 0.03047776222229004 nb_pixel_total : 50125 time to create 1 rle with old method : 0.06380701065063477 time for calcul the mask position with numpy : 0.02438664436340332 nb_pixel_total : 61785 time to create 1 rle with old method : 0.07256174087524414 time for calcul the mask position with numpy : 0.024968385696411133 nb_pixel_total : 5112 time to create 1 rle with old method : 0.006651878356933594 create new chi : 1.7649681568145752 time to delete rle : 0.0011355876922607422 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 8555 TO DO : save crop sub photo not yet done ! save time : 1.00350022315979 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 10.97541069984436 time for calcul the mask position with numpy : 0.3059275150299072 nb_pixel_total : 5904782 time to create 1 rle with new method : 1.9849514961242676 time for calcul the mask position with numpy : 0.029771804809570312 nb_pixel_total : 197543 time to create 1 rle with new method : 2.215538501739502 time for calcul the mask position with numpy : 0.03224921226501465 nb_pixel_total : 59273 time to create 1 rle with old method : 0.06927347183227539 time for calcul the mask position with numpy : 0.03337216377258301 nb_pixel_total : 6341 time to create 1 rle with old method : 0.007616519927978516 time for calcul the mask position with numpy : 0.0328977108001709 nb_pixel_total : 5053 time to create 1 rle with old method : 0.005976676940917969 time for calcul the mask position with numpy : 0.03077864646911621 nb_pixel_total : 1759 time to create 1 rle with old method : 0.002996683120727539 time for calcul the mask position with numpy : 0.03587985038757324 nb_pixel_total : 78843 time to create 1 rle with old method : 0.09227657318115234 time for calcul the mask position with numpy : 0.03609156608581543 nb_pixel_total : 19923 time to create 1 rle with old method : 0.03452658653259277 time for calcul the mask position with numpy : 0.038651227951049805 nb_pixel_total : 51200 time to create 1 rle with old method : 0.0593259334564209 time for calcul the mask position with numpy : 0.035643815994262695 nb_pixel_total : 52183 time to create 1 rle with old method : 0.08371973037719727 time for calcul the mask position with numpy : 0.025132179260253906 nb_pixel_total : 3293 time to create 1 rle with old method : 0.004143476486206055 time for calcul the mask position with numpy : 0.026403427124023438 nb_pixel_total : 380218 time to create 1 rle with new method : 1.2984459400177002 time for calcul the mask position with numpy : 0.025883913040161133 nb_pixel_total : 184707 time to create 1 rle with new method : 0.6965644359588623 time for calcul the mask position with numpy : 0.022050142288208008 nb_pixel_total : 6395 time to create 1 rle with old method : 0.007501363754272461 time for calcul the mask position with numpy : 0.02235555648803711 nb_pixel_total : 61946 time to create 1 rle with old method : 0.07175111770629883 time for calcul the mask position with numpy : 0.02475118637084961 nb_pixel_total : 1745 time to create 1 rle with old method : 0.0020570755004882812 time for calcul the mask position with numpy : 0.022367000579833984 nb_pixel_total : 8465 time to create 1 rle with old method : 0.010073661804199219 time for calcul the mask position with numpy : 0.02996659278869629 nb_pixel_total : 26571 time to create 1 rle with old method : 0.031098604202270508 create new chi : 7.626799583435059 time to delete rle : 0.0017223358154296875 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 11430 TO DO : save crop sub photo not yet done ! save time : 8.905383825302124 nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 9.049670934677124 time for calcul the mask position with numpy : 0.7670497894287109 nb_pixel_total : 6108012 time to create 1 rle with new method : 0.5292437076568604 time for calcul the mask position with numpy : 0.03509783744812012 nb_pixel_total : 34365 time to create 1 rle with old method : 0.03890514373779297 time for calcul the mask position with numpy : 0.03484916687011719 nb_pixel_total : 13728 time to create 1 rle with old method : 0.01615142822265625 time for calcul the mask position with numpy : 0.03764152526855469 nb_pixel_total : 15708 time to create 1 rle with old method : 0.018703937530517578 time for calcul the mask position with numpy : 0.03440976142883301 nb_pixel_total : 24476 time to create 1 rle with old method : 0.028603315353393555 time for calcul the mask position with numpy : 0.03263545036315918 nb_pixel_total : 43585 time to create 1 rle with old method : 0.051985979080200195 time for calcul the mask position with numpy : 0.02701401710510254 nb_pixel_total : 214717 time to create 1 rle with new method : 0.4910545349121094 time for calcul the mask position with numpy : 0.024197101593017578 nb_pixel_total : 58114 time to create 1 rle with old method : 0.06627511978149414 time for calcul the mask position with numpy : 0.024204015731811523 nb_pixel_total : 132320 time to create 1 rle with old method : 0.1551966667175293 time for calcul the mask position with numpy : 0.02340412139892578 nb_pixel_total : 14204 time to create 1 rle with old method : 0.017044782638549805 time for calcul the mask position with numpy : 0.023888111114501953 nb_pixel_total : 218140 time to create 1 rle with new method : 1.143542766571045 time for calcul the mask position with numpy : 0.03692770004272461 nb_pixel_total : 172871 time to create 1 rle with new method : 1.4650921821594238 create new chi : 5.23083233833313 time to delete rle : 0.0011637210845947266 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 11214 TO DO : save crop sub photo not yet done ! save time : 17.827373504638672 nb_obj : 17 nb_hashtags : 7 time to prepare the origin masks : 11.687602519989014 time for calcul the mask position with numpy : 1.4532513618469238 nb_pixel_total : 6386581 time to create 1 rle with new method : 0.7084252834320068 time for calcul the mask position with numpy : 0.03404545783996582 nb_pixel_total : 989 time to create 1 rle with old method : 0.0015687942504882812 time for calcul the mask position with numpy : 0.034682512283325195 nb_pixel_total : 11298 time to create 1 rle with old method : 0.01376485824584961 time for calcul the mask position with numpy : 0.028319358825683594 nb_pixel_total : 92978 time to create 1 rle with old method : 0.10566973686218262 time for calcul the mask position with numpy : 0.02807927131652832 nb_pixel_total : 14663 time to create 1 rle with old method : 0.01835012435913086 time for calcul the mask position with numpy : 0.024493932723999023 nb_pixel_total : 72936 time to create 1 rle with old method : 0.08648347854614258 time for calcul the mask position with numpy : 0.02337336540222168 nb_pixel_total : 10387 time to create 1 rle with old method : 0.011800289154052734 time for calcul the mask position with numpy : 0.02248525619506836 nb_pixel_total : 2809 time to create 1 rle with old method : 0.0033478736877441406 time for calcul the mask position with numpy : 0.021556615829467773 nb_pixel_total : 53103 time to create 1 rle with old method : 0.06093120574951172 time for calcul the mask position with numpy : 0.022950172424316406 nb_pixel_total : 1853 time to create 1 rle with old method : 0.0031049251556396484 time for calcul the mask position with numpy : 0.02457880973815918 nb_pixel_total : 2016 time to create 1 rle with old method : 0.0034058094024658203 time for calcul the mask position with numpy : 0.028124570846557617 nb_pixel_total : 83952 time to create 1 rle with old method : 0.1147463321685791 time for calcul the mask position with numpy : 0.02248692512512207 nb_pixel_total : 7887 time to create 1 rle with old method : 0.00955820083618164 time for calcul the mask position with numpy : 0.022988080978393555 nb_pixel_total : 9799 time to create 1 rle with old method : 0.011261940002441406 time for calcul the mask position with numpy : 0.020856380462646484 nb_pixel_total : 9629 time to create 1 rle with old method : 0.01145029067993164 time for calcul the mask position with numpy : 0.021819353103637695 nb_pixel_total : 87459 time to create 1 rle with old method : 0.10100960731506348 time for calcul the mask position with numpy : 0.024763107299804688 nb_pixel_total : 2051 time to create 1 rle with old method : 0.002597332000732422 time for calcul the mask position with numpy : 0.02738022804260254 nb_pixel_total : 199850 time to create 1 rle with new method : 0.7054603099822998 create new chi : 3.925631284713745 time to delete rle : 0.001102447509765625 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 10315 TO DO : save crop sub photo not yet done ! save time : 17.290510416030884 nb_obj : 17 nb_hashtags : 6 time to prepare the origin masks : 12.14988112449646 time for calcul the mask position with numpy : 0.44013333320617676 nb_pixel_total : 5874794 time to create 1 rle with new method : 0.87125563621521 time for calcul the mask position with numpy : 0.02713155746459961 nb_pixel_total : 44996 time to create 1 rle with old method : 0.07540535926818848 time for calcul the mask position with numpy : 0.03178668022155762 nb_pixel_total : 123399 time to create 1 rle with old method : 0.13892173767089844 time for calcul the mask position with numpy : 0.028896093368530273 nb_pixel_total : 3556 time to create 1 rle with old method : 0.004638195037841797 time for calcul the mask position with numpy : 0.03052377700805664 nb_pixel_total : 4590 time to create 1 rle with old method : 0.006246089935302734 time for calcul the mask position with numpy : 0.038262128829956055 nb_pixel_total : 4145 time to create 1 rle with old method : 0.00791311264038086 time for calcul the mask position with numpy : 0.030145883560180664 nb_pixel_total : 47366 time to create 1 rle with old method : 0.058948516845703125 time for calcul the mask position with numpy : 0.02915215492248535 nb_pixel_total : 164669 time to create 1 rle with new method : 1.9544379711151123 time for calcul the mask position with numpy : 0.026256322860717773 nb_pixel_total : 6129 time to create 1 rle with old method : 0.00665736198425293 time for calcul the mask position with numpy : 0.025171756744384766 nb_pixel_total : 47373 time to create 1 rle with old method : 0.06861090660095215 time for calcul the mask position with numpy : 0.031599998474121094 nb_pixel_total : 460218 time to create 1 rle with new method : 0.6031308174133301 time for calcul the mask position with numpy : 0.02904534339904785 nb_pixel_total : 5917 time to create 1 rle with old method : 0.010290384292602539 time for calcul the mask position with numpy : 0.033257484436035156 nb_pixel_total : 50485 time to create 1 rle with old method : 0.06760191917419434 time for calcul the mask position with numpy : 0.027075529098510742 nb_pixel_total : 95228 time to create 1 rle with old method : 0.11978268623352051 time for calcul the mask position with numpy : 0.04239392280578613 nb_pixel_total : 53613 time to create 1 rle with old method : 0.060793399810791016 time for calcul the mask position with numpy : 0.0498507022857666 nb_pixel_total : 7077 time to create 1 rle with old method : 0.008283615112304688 time for calcul the mask position with numpy : 0.04220914840698242 nb_pixel_total : 3051 time to create 1 rle with old method : 0.0036711692810058594 time for calcul the mask position with numpy : 0.04282426834106445 nb_pixel_total : 53634 time to create 1 rle with old method : 0.062294721603393555 create new chi : 5.250197887420654 time to delete rle : 0.0013272762298583984 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 11773 TO DO : save crop sub photo not yet done ! save time : 15.471311330795288 nb_obj : 28 nb_hashtags : 7 time to prepare the origin masks : 13.555763006210327 time for calcul the mask position with numpy : 0.5952606201171875 nb_pixel_total : 5703736 time to create 1 rle with new method : 0.6990783214569092 time for calcul the mask position with numpy : 0.030736207962036133 nb_pixel_total : 8702 time to create 1 rle with old method : 0.010505437850952148 time for calcul the mask position with numpy : 0.031285762786865234 nb_pixel_total : 108999 time to create 1 rle with old method : 0.12927794456481934 time for calcul the mask position with numpy : 0.029558897018432617 nb_pixel_total : 2307 time to create 1 rle with old method : 0.0026166439056396484 time for calcul the mask position with numpy : 0.033812761306762695 nb_pixel_total : 226176 time to create 1 rle with new method : 0.8185031414031982 time for calcul the mask position with numpy : 0.02927565574645996 nb_pixel_total : 3684 time to create 1 rle with old method : 0.004610300064086914 time for calcul the mask position with numpy : 0.030052661895751953 nb_pixel_total : 182437 time to create 1 rle with new method : 0.40468931198120117 time for calcul the mask position with numpy : 0.02878546714782715 nb_pixel_total : 84835 time to create 1 rle with old method : 0.10074448585510254 time for calcul the mask position with numpy : 0.02948164939880371 nb_pixel_total : 14268 time to create 1 rle with old method : 0.017413854598999023 time for calcul the mask position with numpy : 0.029718637466430664 nb_pixel_total : 31319 time to create 1 rle with old method : 0.036566734313964844 time for calcul the mask position with numpy : 0.02898550033569336 nb_pixel_total : 2989 time to create 1 rle with old method : 0.0038046836853027344 time for calcul the mask position with numpy : 0.02881908416748047 nb_pixel_total : 23604 time to create 1 rle with old method : 0.027338504791259766 time for calcul the mask position with numpy : 0.02929234504699707 nb_pixel_total : 9183 time to create 1 rle with old method : 0.010888099670410156 time for calcul the mask position with numpy : 0.028815031051635742 nb_pixel_total : 5563 time to create 1 rle with old method : 0.006566286087036133 time for calcul the mask position with numpy : 0.02964496612548828 nb_pixel_total : 99823 time to create 1 rle with old method : 0.11123800277709961 time for calcul the mask position with numpy : 0.028703689575195312 nb_pixel_total : 13106 time to create 1 rle with old method : 0.015454292297363281 time for calcul the mask position with numpy : 0.028508663177490234 nb_pixel_total : 66359 time to create 1 rle with old method : 0.07682132720947266 time for calcul the mask position with numpy : 0.028539180755615234 nb_pixel_total : 149149 time to create 1 rle with old method : 0.17026352882385254 time for calcul the mask position with numpy : 0.027979612350463867 nb_pixel_total : 39311 time to create 1 rle with old method : 0.042611122131347656 time for calcul the mask position with numpy : 0.027269363403320312 nb_pixel_total : 68697 time to create 1 rle with old method : 0.07648539543151855 time for calcul the mask position with numpy : 0.02854299545288086 nb_pixel_total : 81792 time to create 1 rle with old method : 0.13056159019470215 time for calcul the mask position with numpy : 0.028484344482421875 nb_pixel_total : 7427 time to create 1 rle with old method : 0.008704423904418945 time for calcul the mask position with numpy : 0.028675317764282227 nb_pixel_total : 10467 time to create 1 rle with old method : 0.012490034103393555 time for calcul the mask position with numpy : 0.02999424934387207 nb_pixel_total : 25086 time to create 1 rle with old method : 0.02716827392578125 time for calcul the mask position with numpy : 0.02672743797302246 nb_pixel_total : 46756 time to create 1 rle with old method : 0.05166816711425781 time for calcul the mask position with numpy : 0.02787494659423828 nb_pixel_total : 13484 time to create 1 rle with old method : 0.014945507049560547 time for calcul the mask position with numpy : 0.027055978775024414 nb_pixel_total : 9358 time to create 1 rle with old method : 0.010677576065063477 time for calcul the mask position with numpy : 0.027669191360473633 nb_pixel_total : 10721 time to create 1 rle with old method : 0.012378454208374023 time for calcul the mask position with numpy : 0.02747654914855957 nb_pixel_total : 902 time to create 1 rle with old method : 0.0010821819305419922 create new chi : 4.543035984039307 time to delete rle : 0.001767873764038086 batch 1 Loaded 29 chid ids of type : 3726 Number RLEs to save : 18312 TO DO : save crop sub photo not yet done ! save time : 26.643248796463013 nb_obj : 25 nb_hashtags : 8 time to prepare the origin masks : 19.907630443572998 time for calcul the mask position with numpy : 0.497697114944458 nb_pixel_total : 5888805 time to create 1 rle with new method : 0.5672595500946045 time for calcul the mask position with numpy : 0.02620387077331543 nb_pixel_total : 242 time to create 1 rle with old method : 0.0005404949188232422 time for calcul the mask position with numpy : 0.023720502853393555 nb_pixel_total : 131539 time to create 1 rle with old method : 0.1912844181060791 time for calcul the mask position with numpy : 0.0227658748626709 nb_pixel_total : 61849 time to create 1 rle with old method : 0.06863260269165039 time for calcul the mask position with numpy : 0.03098273277282715 nb_pixel_total : 3983 time to create 1 rle with old method : 0.004785060882568359 time for calcul the mask position with numpy : 0.029999732971191406 nb_pixel_total : 73509 time to create 1 rle with old method : 0.1094515323638916 time for calcul the mask position with numpy : 0.02480936050415039 nb_pixel_total : 813 time to create 1 rle with old method : 0.001024484634399414 time for calcul the mask position with numpy : 0.022172212600708008 nb_pixel_total : 49297 time to create 1 rle with old method : 0.06331276893615723 time for calcul the mask position with numpy : 0.03931593894958496 nb_pixel_total : 1483 time to create 1 rle with old method : 0.0018036365509033203 time for calcul the mask position with numpy : 0.040863037109375 nb_pixel_total : 45865 time to create 1 rle with old method : 0.05712747573852539 time for calcul the mask position with numpy : 0.04237031936645508 nb_pixel_total : 36825 time to create 1 rle with old method : 0.05664849281311035 time for calcul the mask position with numpy : 0.0405735969543457 nb_pixel_total : 107125 time to create 1 rle with old method : 0.13176918029785156 time for calcul the mask position with numpy : 0.04070544242858887 nb_pixel_total : 75964 time to create 1 rle with old method : 0.091033935546875 time for calcul the mask position with numpy : 0.03596019744873047 nb_pixel_total : 29100 time to create 1 rle with old method : 0.03616929054260254 time for calcul the mask position with numpy : 0.03833746910095215 nb_pixel_total : 45757 time to create 1 rle with old method : 0.06005144119262695 time for calcul the mask position with numpy : 0.03570222854614258 nb_pixel_total : 4568 time to create 1 rle with old method : 0.005268096923828125 time for calcul the mask position with numpy : 0.035817623138427734 nb_pixel_total : 197188 time to create 1 rle with new method : 0.8029179573059082 time for calcul the mask position with numpy : 0.03263425827026367 nb_pixel_total : 11361 time to create 1 rle with old method : 0.012176513671875 time for calcul the mask position with numpy : 0.03278064727783203 nb_pixel_total : 22234 time to create 1 rle with old method : 0.023688316345214844 time for calcul the mask position with numpy : 0.03343534469604492 nb_pixel_total : 37649 time to create 1 rle with old method : 0.03974795341491699 time for calcul the mask position with numpy : 0.03366971015930176 nb_pixel_total : 60953 time to create 1 rle with old method : 0.07246685028076172 time for calcul the mask position with numpy : 0.04068422317504883 nb_pixel_total : 17812 time to create 1 rle with old method : 0.022562265396118164 time for calcul the mask position with numpy : 0.03804969787597656 nb_pixel_total : 75800 time to create 1 rle with old method : 0.08683562278747559 time for calcul the mask position with numpy : 0.0437319278717041 nb_pixel_total : 3544 time to create 1 rle with old method : 0.00444793701171875 time for calcul the mask position with numpy : 0.03961324691772461 nb_pixel_total : 41158 time to create 1 rle with old method : 0.05033063888549805 time for calcul the mask position with numpy : 0.034540653228759766 nb_pixel_total : 25817 time to create 1 rle with old method : 0.02983403205871582 create new chi : 4.0121355056762695 time to delete rle : 0.0022492408752441406 batch 1 Loaded 26 chid ids of type : 3726 Number RLEs to save : 16658 TO DO : save crop sub photo not yet done ! save time : 10.55882215499878 nb_obj : 28 nb_hashtags : 7 time to prepare the origin masks : 11.515452861785889 time for calcul the mask position with numpy : 0.5902988910675049 nb_pixel_total : 5425159 time to create 1 rle with new method : 0.5921430587768555 time for calcul the mask position with numpy : 0.029253721237182617 nb_pixel_total : 50596 time to create 1 rle with old method : 0.06243395805358887 time for calcul the mask position with numpy : 0.028874874114990234 nb_pixel_total : 90958 time to create 1 rle with old method : 0.10847353935241699 time for calcul the mask position with numpy : 0.029107332229614258 nb_pixel_total : 27777 time to create 1 rle with old method : 0.03245043754577637 time for calcul the mask position with numpy : 0.029075145721435547 nb_pixel_total : 2591 time to create 1 rle with old method : 0.0031480789184570312 time for calcul the mask position with numpy : 0.03679370880126953 nb_pixel_total : 462238 time to create 1 rle with new method : 0.41893482208251953 time for calcul the mask position with numpy : 0.03225111961364746 nb_pixel_total : 102803 time to create 1 rle with old method : 0.1203012466430664 time for calcul the mask position with numpy : 0.029290437698364258 nb_pixel_total : 801 time to create 1 rle with old method : 0.0011429786682128906 time for calcul the mask position with numpy : 0.02955007553100586 nb_pixel_total : 22597 time to create 1 rle with old method : 0.02677011489868164 time for calcul the mask position with numpy : 0.03320169448852539 nb_pixel_total : 37159 time to create 1 rle with old method : 0.04783749580383301 time for calcul the mask position with numpy : 0.035236358642578125 nb_pixel_total : 9071 time to create 1 rle with old method : 0.012682676315307617 time for calcul the mask position with numpy : 0.0345921516418457 nb_pixel_total : 17829 time to create 1 rle with old method : 0.0211489200592041 time for calcul the mask position with numpy : 0.03033447265625 nb_pixel_total : 1382 time to create 1 rle with old method : 0.0017323493957519531 time for calcul the mask position with numpy : 0.030233383178710938 nb_pixel_total : 108133 time to create 1 rle with old method : 0.1259019374847412 time for calcul the mask position with numpy : 0.03020167350769043 nb_pixel_total : 128268 time to create 1 rle with old method : 0.1628255844116211 time for calcul the mask position with numpy : 0.03624272346496582 nb_pixel_total : 52375 time to create 1 rle with old method : 0.0748450756072998 time for calcul the mask position with numpy : 0.029492616653442383 nb_pixel_total : 6171 time to create 1 rle with old method : 0.008056640625 time for calcul the mask position with numpy : 0.029633283615112305 nb_pixel_total : 48180 time to create 1 rle with old method : 0.056270599365234375 time for calcul the mask position with numpy : 0.029216527938842773 nb_pixel_total : 24920 time to create 1 rle with old method : 0.0297393798828125 time for calcul the mask position with numpy : 0.03237128257751465 nb_pixel_total : 1732 time to create 1 rle with old method : 0.0024569034576416016 time for calcul the mask position with numpy : 0.039788007736206055 nb_pixel_total : 50297 time to create 1 rle with old method : 0.07697129249572754 time for calcul the mask position with numpy : 0.03496098518371582 nb_pixel_total : 3028 time to create 1 rle with old method : 0.004070758819580078 time for calcul the mask position with numpy : 0.03226828575134277 nb_pixel_total : 8358 time to create 1 rle with old method : 0.012017011642456055 time for calcul the mask position with numpy : 0.03148841857910156 nb_pixel_total : 161893 time to create 1 rle with new method : 0.8029730319976807 time for calcul the mask position with numpy : 0.030401229858398438 nb_pixel_total : 20908 time to create 1 rle with old method : 0.027392148971557617 time for calcul the mask position with numpy : 0.03090524673461914 nb_pixel_total : 98791 time to create 1 rle with old method : 0.11909246444702148 time for calcul the mask position with numpy : 0.04713916778564453 nb_pixel_total : 71553 time to create 1 rle with old method : 0.09322094917297363 time for calcul the mask position with numpy : 0.032716989517211914 nb_pixel_total : 8370 time to create 1 rle with old method : 0.010773420333862305 time for calcul the mask position with numpy : 0.030670166015625 nb_pixel_total : 6302 time to create 1 rle with old method : 0.007428646087646484 create new chi : 4.670834541320801 time to delete rle : 0.0019147396087646484 batch 1 Loaded 29 chid ids of type : 3726 Number RLEs to save : 19044 TO DO : save crop sub photo not yet done ! save time : 15.152236938476562 map_output_result : {1343023136: (0.0, 'Should be the crop_list due to order', 0), 1343023075: (0.0, 'Should be the crop_list due to order', 0), 1343022775: (0.0, 'Should be the crop_list due to order', 0), 1343022749: (0.0, 'Should be the crop_list due to order', 0), 1343022747: (0.0, 'Should be the crop_list due to order', 0), 1343022739: (0.0, 'Should be the crop_list due to order', 0), 1343022531: (0.0, 'Should be the crop_list due to order', 0), 1343022217: (0.0, 'Should be the crop_list due to order', 0), 1343022175: (0.0, 'Should be the crop_list due to order', 0), 1343022101: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 10 /1343023136.Didn't retrieve data . /1343023075.Didn't retrieve data . /1343022775.Didn't retrieve data . /1343022749.Didn't retrieve data . /1343022747.Didn't retrieve data . /1343022739.Didn't retrieve data . /1343022531.Didn't retrieve data . /1343022217.Didn't retrieve data . /1343022175.Didn't retrieve data . /1343022101.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023136', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023075', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022775', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022749', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022747', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022739', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022531', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022217', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022175', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022101', None, None, None, None, None, '2658419') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.7344293594360352 save_final save missing photos in datou_result : time spend for datou_step_exec : 294.2808041572571 time spend to save output : 0.7361621856689453 total time spend for step 5 : 295.016966342926 step6:crop_condition Tue Mar 11 15:05:05 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 205 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! About to upload 19 photos upload in portfolio : 4869462 init cache_photo without model_param we have 19 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741701913_1309421 we have uploaded 19 photos in the portfolio 4869462 time of upload the photos Elapsed time : 6.44803786277771 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741701925_1309421 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.469062089920044 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741701930_1309421 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.5667729377746582 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741701934_1309421 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.415231943130493 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741701941_1309421 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.367000579833984 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741701948_1309421 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 6.824323415756226 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741701958_1309421 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.0845673084259033 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 50 /1343061080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061188Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061190Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061244Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343061333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023136', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023075', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022775', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022749', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022747', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022739', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022531', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022217', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022175', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022101', None, None, None, None, None, '2658419') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 160 time used for this insertion : 0.27432823181152344 save_final save missing photos in datou_result : time spend for datou_step_exec : 61.50400519371033 time spend to save output : 0.29588770866394043 total time spend for step 6 : 61.79989290237427 step7:ventilate_hashtags_in_portfolio Tue Mar 11 15:06:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 21290053 get user id for portfolio 21290053 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290053 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','metal','kraft','Teint_Dans_La_Masse','flou','plastique','cartonnette','papier','mal_croppe','environnement','autre_refus','Carton_gris')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290053 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','metal','kraft','Teint_Dans_La_Masse','flou','plastique','cartonnette','papier','mal_croppe','environnement','autre_refus','Carton_gris')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290053 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','metal','kraft','Teint_Dans_La_Masse','flou','plastique','cartonnette','papier','mal_croppe','environnement','autre_refus','Carton_gris')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/21290856,21290857,21290858,21290859,21290860,21290861,21290862,21290863,21290864,21290865,21290866,21290867?tags=Carton_brun,metal,kraft,Teint_Dans_La_Masse,flou,plastique,cartonnette,papier,mal_croppe,environnement,autre_refus,Carton_gris Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 1 /21290053. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023136', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023075', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022775', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022749', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022747', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022739', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022531', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022217', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022175', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022101', None, None, None, None, None, '2658419') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.025849580764770508 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.9578769207000732 time spend to save output : 0.026236772537231445 total time spend for step 7 : 1.9841136932373047 step8:final Tue Mar 11 15:06:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1343023136: ('0.07704934125142994',), 1343023075: ('0.07704934125142994',), 1343022775: ('0.07704934125142994',), 1343022749: ('0.07704934125142994',), 1343022747: ('0.07704934125142994',), 1343022739: ('0.07704934125142994',), 1343022531: ('0.07704934125142994',), 1343022217: ('0.07704934125142994',), 1343022175: ('0.07704934125142994',), 1343022101: ('0.07704934125142994',)} new output for save of step final : {1343023136: ('0.07704934125142994',), 1343023075: ('0.07704934125142994',), 1343022775: ('0.07704934125142994',), 1343022749: ('0.07704934125142994',), 1343022747: ('0.07704934125142994',), 1343022739: ('0.07704934125142994',), 1343022531: ('0.07704934125142994',), 1343022217: ('0.07704934125142994',), 1343022175: ('0.07704934125142994',), 1343022101: ('0.07704934125142994',)} [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 10 /1343023136.Didn't retrieve data . /1343023075.Didn't retrieve data . /1343022775.Didn't retrieve data . /1343022749.Didn't retrieve data . /1343022747.Didn't retrieve data . /1343022739.Didn't retrieve data . /1343022531.Didn't retrieve data . /1343022217.Didn't retrieve data . /1343022175.Didn't retrieve data . /1343022101.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023136', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023075', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022775', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022749', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022747', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022739', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022531', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022217', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022175', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022101', None, None, None, None, None, '2658419') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013343095779418945 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10884380340576172 time spend to save output : 0.013968706130981445 total time spend for step 8 : 0.12281250953674316 step9:velours_tree Tue Mar 11 15:06:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.1753537654876709 time spend to save output : 5.2928924560546875e-05 total time spend for step 9 : 0.17540669441223145 step10:send_mail_cod Tue Mar 11 15:06:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P21290053_11-03-2025_15_06_09.pdf 21290857 imagette212908571741701969 21290858 change filename to text .change filename to text .change filename to text .imagette212908581741701969 21290860 imagette212908601741701970 21290861 change filename to text .change filename to text .change filename to text .imagette212908611741701970 21290862 change filename to text .change filename to text .change filename to text .imagette212908621741701970 21290864 imagette212908641741701970 21290866 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212908661741701970 21290856 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212908561741701971 21290859 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212908591741701971 21290867 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212908671741701972 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21290053 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21290856,21290857,21290858,21290859,21290860,21290861,21290862,21290863,21290864,21290865,21290866,21290867?tags=Carton_brun,metal,kraft,Teint_Dans_La_Masse,flou,plastique,cartonnette,papier,mal_croppe,environnement,autre_refus,Carton_gris your option no_mail is active, we will not send the real mail to your client args[1343023136] : ('0.07704934125142994',) no score found for photo 1343023136 We are sending mail with results at report@fotonower.com args[1343023075] : ('0.07704934125142994',) no score found for photo 1343023075 We are sending mail with results at report@fotonower.com args[1343022775] : ('0.07704934125142994',) no score found for photo 1343022775 We are sending mail with results at report@fotonower.com args[1343022749] : ('0.07704934125142994',) no score found for photo 1343022749 We are sending mail with results at report@fotonower.com args[1343022747] : ('0.07704934125142994',) no score found for photo 1343022747 We are sending mail with results at report@fotonower.com args[1343022739] : ('0.07704934125142994',) no score found for photo 1343022739 We are sending mail with results at report@fotonower.com args[1343022531] : ('0.07704934125142994',) no score found for photo 1343022531 We are sending mail with results at report@fotonower.com args[1343022217] : ('0.07704934125142994',) no score found for photo 1343022217 We are sending mail with results at report@fotonower.com args[1343022175] : ('0.07704934125142994',) no score found for photo 1343022175 We are sending mail with results at report@fotonower.com args[1343022101] : ('0.07704934125142994',) no score found for photo 1343022101 We are sending mail with results at report@fotonower.com refus_total : 0.07704934125142994 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21290053 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1343021377,1343021425,1343021430,1343022749,1343022747,1343021654,1343021678,1343022739,1343022531,1343022217,1343019725,1343023075,1343022775,1343022175,1343022053,1343023136,1343022101,1343022083,1343019356,1343021603) Found this number of photos: 20 begin to download photo : 1343021377 begin to download photo : 1343021654 begin to download photo : 1343019725 begin to download photo : 1343023136 download finish for photo 1343021377 begin to download photo : 1343021425 download finish for photo 1343019725 begin to download photo : 1343023075 download finish for photo 1343021654 begin to download photo : 1343021678 download finish for photo 1343023136 begin to download photo : 1343022101 download finish for photo 1343023075 begin to download photo : 1343022775 download finish for photo 1343021678 begin to download photo : 1343022739 download finish for photo 1343021425 begin to download photo : 1343021430 download finish for photo 1343022775 begin to download photo : 1343022175 download finish for photo 1343021430 begin to download photo : 1343022749 download finish for photo 1343022101 begin to download photo : 1343022083 download finish for photo 1343022739 begin to download photo : 1343022531 download finish for photo 1343022175 begin to download photo : 1343022053 download finish for photo 1343022083 begin to download photo : 1343019356 download finish for photo 1343022531 begin to download photo : 1343022217 download finish for photo 1343022749 begin to download photo : 1343022747 download finish for photo 1343019356 begin to download photo : 1343021603 download finish for photo 1343022053 download finish for photo 1343022217 download finish for photo 1343022747 download finish for photo 1343021603 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290053_11-03-2025_15_06_09.pdf results_Auto_P21290053_11-03-2025_15_06_09.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290053_11-03-2025_15_06_09.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','21290053','results_Auto_P21290053_11-03-2025_15_06_09.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290053_11-03-2025_15_06_09.pdf','pdf','','0.8','0.07704934125142994') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023136', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023075', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022775', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022749', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022747', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022739', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022531', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022217', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022175', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022101', None, None, None, None, None, '2658419') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.014320135116577148 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.498739719390869 time spend to save output : 0.014593362808227539 total time spend for step 10 : 7.513333082199097 step11:split_time_score Tue Mar 11 15:06:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('13', 24),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11032025 21290053 Nombre de photos uploadées : 24 / 23040 (0%) 11032025 21290053 Nombre de photos taguées (types de déchets): 0 / 24 (0%) 11032025 21290053 Nombre de photos taguées (volume) : 0 / 24 (0%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 8.821487426757812e-06 ???????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0016570091247558594 elapsed_time : insert_dashboard_record_day_entry 0.022157907485961914 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.07848604537653218 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268995_11-03-2025_10_04_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268995 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268995 AND mptpi.`type`=4207 To do Qualite : 0.21508235889842053 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268996_11-03-2025_09_21_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268996 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268996 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268997_11-03-2025_09_36_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268997 order by id desc limit 1 Qualite : 0.2996323014615577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268998_11-03-2025_09_21_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268998 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268998 AND mptpi.`type`=4200 To do Qualite : 0.059107923268759154 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21269000_11-03-2025_09_44_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21269000 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21269000 AND mptpi.`type`=4209 To do Qualite : 0.09529821171173529 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21269001_11-03-2025_09_17_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21269001 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21269001 AND mptpi.`type`=3726 To do Qualite : 0.12370809963529884 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21270945_11-03-2025_12_54_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21270945 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21270945 AND mptpi.`type`=3327 To do Qualite : 0.044100679011939585 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21276914_11-03-2025_11_38_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21276914 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21276914 AND mptpi.`type`=4207 To do Qualite : 0.04798165055335807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21276915_11-03-2025_12_41_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21276915 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21276915 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290022 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290047 order by id desc limit 1 Qualite : 0.24539234952758107 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282296_11-03-2025_13_26_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282296 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282296 AND mptpi.`type`=3594 To do Qualite : 0.23651912322606505 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282297_11-03-2025_14_28_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282297 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282297 AND mptpi.`type`=3594 To do Qualite : 0.30571381203013953 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282298_11-03-2025_13_24_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282298 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282298 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282299_11-03-2025_12_43_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282299 order by id desc limit 1 Qualite : 0.3846979317267506 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282300_11-03-2025_13_27_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282300 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282300 AND mptpi.`type`=4200 To do Qualite : 0.3844764824714978 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282301_11-03-2025_13_46_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282301 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282301 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282303_11-03-2025_12_44_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282303 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290048 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290049 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290050 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290051 order by id desc limit 1 Qualite : 0.04833780284360806 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290052_11-03-2025_15_05_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290052 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290052 AND mptpi.`type`=4207 To do Qualite : 0.07704934125142994 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290053_11-03-2025_15_06_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290053 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290053 AND mptpi.`type`=3726 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11032025': {'nb_upload': 24, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 1 /21290053Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023136', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343023075', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022775', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022749', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022747', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022739', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022531', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022217', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022175', None, None, None, None, None, '2658419') ('3459', None, None, None, None, None, None, None, '2658419') ('3459', '21290053', '1343022101', None, None, None, None, None, '2658419') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.013066768646240234 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.0280680656433105 time spend to save output : 0.013351917266845703 total time spend for step 11 : 7.041419982910156 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 208.68user 155.52system 10:02.27elapsed 60%CPU (0avgtext+0avgdata 4991568maxresident)k 382456inputs+144096outputs (20640major+16012711minor)pagefaults 0swaps