python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1396219 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 20>10 please execute split_portfolio.py -i 21291428 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2658518'] with mtr_portfolio_ids : ['21291428'] and first list_photo_ids : [] new path : /proc/1396219/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.6725094318389893 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Mar 11 15:20:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-11 15:20:34.651886: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-11 15:20:34.679070: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-11 15:20:34.680907: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f30b0000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-11 15:20:34.680940: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-11 15:20:34.684999: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-11 15:20:34.832880: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5bafcb0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-11 15:20:34.832939: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-11 15:20:34.834324: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 15:20:34.834749: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 15:20:34.837628: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 15:20:34.840757: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 15:20:34.841444: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 15:20:34.844760: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 15:20:34.846358: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 15:20:34.852941: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 15:20:34.855029: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 15:20:34.855143: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 15:20:34.856093: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 15:20:34.856112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 15:20:34.856126: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 15:20:34.857689: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9879 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-11 15:20:35.246266: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 15:20:35.246420: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 15:20:35.246454: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 15:20:35.246485: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 15:20:35.246514: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 15:20:35.246542: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 15:20:35.246574: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 15:20:35.246607: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 15:20:35.248539: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 15:20:35.250231: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 15:20:35.250341: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 15:20:35.250381: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 15:20:35.250412: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 15:20:35.250445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 15:20:35.250472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 15:20:35.250501: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 15:20:35.250531: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 15:20:35.252402: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 15:20:35.252455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 15:20:35.252471: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 15:20:35.252484: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 15:20:35.254401: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9879 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-11 15:20:46.692600: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 15:20:46.885863: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 35 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.63302 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 34 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 28 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 34 Detection mask done ! Trying to reset tf kernel 1397178 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 555 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9621 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0008227825164794922 nb_pixel_total : 15267 time to create 1 rle with old method : 0.023671865463256836 length of segment : 138 time for calcul the mask position with numpy : 0.40682172775268555 nb_pixel_total : 3760235 time to create 1 rle with new method : 0.5527095794677734 length of segment : 3552 time for calcul the mask position with numpy : 0.0012197494506835938 nb_pixel_total : 91883 time to create 1 rle with old method : 0.10099911689758301 length of segment : 360 time for calcul the mask position with numpy : 0.0003788471221923828 nb_pixel_total : 8843 time to create 1 rle with old method : 0.01453709602355957 length of segment : 253 time for calcul the mask position with numpy : 0.0004172325134277344 nb_pixel_total : 14165 time to create 1 rle with old method : 0.016640424728393555 length of segment : 120 time for calcul the mask position with numpy : 0.0025675296783447266 nb_pixel_total : 168203 time to create 1 rle with new method : 0.004528999328613281 length of segment : 354 time for calcul the mask position with numpy : 0.0008559226989746094 nb_pixel_total : 38709 time to create 1 rle with old method : 0.0437774658203125 length of segment : 415 time for calcul the mask position with numpy : 0.00018930435180664062 nb_pixel_total : 2916 time to create 1 rle with old method : 0.003815174102783203 length of segment : 63 time for calcul the mask position with numpy : 0.00015926361083984375 nb_pixel_total : 3516 time to create 1 rle with old method : 0.004564046859741211 length of segment : 44 time for calcul the mask position with numpy : 0.005146026611328125 nb_pixel_total : 169612 time to create 1 rle with new method : 0.008246421813964844 length of segment : 451 time for calcul the mask position with numpy : 0.0008008480072021484 nb_pixel_total : 16920 time to create 1 rle with old method : 0.020374774932861328 length of segment : 147 time for calcul the mask position with numpy : 0.002574920654296875 nb_pixel_total : 67841 time to create 1 rle with old method : 0.0781393051147461 length of segment : 454 time for calcul the mask position with numpy : 0.0033621788024902344 nb_pixel_total : 94754 time to create 1 rle with old method : 0.10785698890686035 length of segment : 294 time for calcul the mask position with numpy : 0.0006573200225830078 nb_pixel_total : 12008 time to create 1 rle with old method : 0.014681339263916016 length of segment : 147 time for calcul the mask position with numpy : 0.00019550323486328125 nb_pixel_total : 1791 time to create 1 rle with old method : 0.002327442169189453 length of segment : 60 time for calcul the mask position with numpy : 0.0027704238891601562 nb_pixel_total : 46435 time to create 1 rle with old method : 0.05530071258544922 length of segment : 362 time for calcul the mask position with numpy : 0.0023343563079833984 nb_pixel_total : 50834 time to create 1 rle with old method : 0.07902717590332031 length of segment : 416 time for calcul the mask position with numpy : 0.00017881393432617188 nb_pixel_total : 1779 time to create 1 rle with old method : 0.002281665802001953 length of segment : 54 time for calcul the mask position with numpy : 0.0020873546600341797 nb_pixel_total : 50877 time to create 1 rle with old method : 0.059702396392822266 length of segment : 320 time for calcul the mask position with numpy : 0.0018453598022460938 nb_pixel_total : 38568 time to create 1 rle with old method : 0.04643821716308594 length of segment : 267 time for calcul the mask position with numpy : 0.004617214202880859 nb_pixel_total : 151948 time to create 1 rle with new method : 0.0069577693939208984 length of segment : 408 time for calcul the mask position with numpy : 0.0010859966278076172 nb_pixel_total : 20932 time to create 1 rle with old method : 0.0255582332611084 length of segment : 178 time for calcul the mask position with numpy : 0.0015985965728759766 nb_pixel_total : 47815 time to create 1 rle with old method : 0.059099435806274414 length of segment : 149 time for calcul the mask position with numpy : 0.0015115737915039062 nb_pixel_total : 34705 time to create 1 rle with old method : 0.04176616668701172 length of segment : 228 time for calcul the mask position with numpy : 0.005413055419921875 nb_pixel_total : 129758 time to create 1 rle with old method : 0.14141297340393066 length of segment : 670 time for calcul the mask position with numpy : 0.002311229705810547 nb_pixel_total : 53074 time to create 1 rle with old method : 0.062293291091918945 length of segment : 329 time for calcul the mask position with numpy : 0.0070133209228515625 nb_pixel_total : 41017 time to create 1 rle with old method : 0.04684567451477051 length of segment : 654 time for calcul the mask position with numpy : 0.0004563331604003906 nb_pixel_total : 4698 time to create 1 rle with old method : 0.005402088165283203 length of segment : 99 time for calcul the mask position with numpy : 0.0015158653259277344 nb_pixel_total : 33534 time to create 1 rle with old method : 0.03863954544067383 length of segment : 206 time for calcul the mask position with numpy : 0.0009212493896484375 nb_pixel_total : 28749 time to create 1 rle with old method : 0.032372474670410156 length of segment : 224 time for calcul the mask position with numpy : 0.0025451183319091797 nb_pixel_total : 43213 time to create 1 rle with old method : 0.05193686485290527 length of segment : 149 time for calcul the mask position with numpy : 0.0007658004760742188 nb_pixel_total : 12612 time to create 1 rle with old method : 0.015149354934692383 length of segment : 175 time for calcul the mask position with numpy : 0.0006742477416992188 nb_pixel_total : 30635 time to create 1 rle with old method : 0.034485816955566406 length of segment : 247 time for calcul the mask position with numpy : 0.0006175041198730469 nb_pixel_total : 26188 time to create 1 rle with old method : 0.031190872192382812 length of segment : 261 time for calcul the mask position with numpy : 0.0011806488037109375 nb_pixel_total : 25154 time to create 1 rle with old method : 0.02959728240966797 length of segment : 212 time for calcul the mask position with numpy : 0.0008504390716552734 nb_pixel_total : 17085 time to create 1 rle with old method : 0.020835161209106445 length of segment : 147 time for calcul the mask position with numpy : 0.004743099212646484 nb_pixel_total : 103382 time to create 1 rle with old method : 0.11909914016723633 length of segment : 288 time for calcul the mask position with numpy : 0.0012874603271484375 nb_pixel_total : 19341 time to create 1 rle with old method : 0.02971482276916504 length of segment : 188 time for calcul the mask position with numpy : 0.004300594329833984 nb_pixel_total : 81951 time to create 1 rle with old method : 0.09604239463806152 length of segment : 474 time for calcul the mask position with numpy : 0.0046596527099609375 nb_pixel_total : 92801 time to create 1 rle with old method : 0.10897493362426758 length of segment : 655 time for calcul the mask position with numpy : 0.0007379055023193359 nb_pixel_total : 28543 time to create 1 rle with old method : 0.0340273380279541 length of segment : 247 time for calcul the mask position with numpy : 0.0010499954223632812 nb_pixel_total : 19000 time to create 1 rle with old method : 0.022382736206054688 length of segment : 183 time for calcul the mask position with numpy : 0.0027751922607421875 nb_pixel_total : 103548 time to create 1 rle with old method : 0.12050008773803711 length of segment : 560 time for calcul the mask position with numpy : 0.003292560577392578 nb_pixel_total : 116322 time to create 1 rle with old method : 0.13369512557983398 length of segment : 593 time for calcul the mask position with numpy : 0.0004146099090576172 nb_pixel_total : 5119 time to create 1 rle with old method : 0.006369829177856445 length of segment : 133 time for calcul the mask position with numpy : 0.0016379356384277344 nb_pixel_total : 41700 time to create 1 rle with old method : 0.04814743995666504 length of segment : 333 time for calcul the mask position with numpy : 0.0016524791717529297 nb_pixel_total : 67928 time to create 1 rle with old method : 0.07436609268188477 length of segment : 286 time for calcul the mask position with numpy : 0.0004596710205078125 nb_pixel_total : 14088 time to create 1 rle with old method : 0.017053842544555664 length of segment : 149 time for calcul the mask position with numpy : 0.0010304450988769531 nb_pixel_total : 50360 time to create 1 rle with old method : 0.060391902923583984 length of segment : 233 time for calcul the mask position with numpy : 0.0019884109497070312 nb_pixel_total : 61881 time to create 1 rle with old method : 0.07172012329101562 length of segment : 315 time for calcul the mask position with numpy : 0.0003025531768798828 nb_pixel_total : 4554 time to create 1 rle with old method : 0.0056874752044677734 length of segment : 80 time for calcul the mask position with numpy : 0.0013570785522460938 nb_pixel_total : 30393 time to create 1 rle with old method : 0.034790754318237305 length of segment : 241 time for calcul the mask position with numpy : 0.00024271011352539062 nb_pixel_total : 3799 time to create 1 rle with old method : 0.004573822021484375 length of segment : 104 time for calcul the mask position with numpy : 0.0002307891845703125 nb_pixel_total : 5060 time to create 1 rle with old method : 0.0064542293548583984 length of segment : 61 time for calcul the mask position with numpy : 0.0005724430084228516 nb_pixel_total : 15496 time to create 1 rle with old method : 0.01850271224975586 length of segment : 298 time for calcul the mask position with numpy : 0.0006306171417236328 nb_pixel_total : 47674 time to create 1 rle with old method : 0.05352449417114258 length of segment : 236 time for calcul the mask position with numpy : 0.0010831356048583984 nb_pixel_total : 26621 time to create 1 rle with old method : 0.031123876571655273 length of segment : 338 time for calcul the mask position with numpy : 0.0001621246337890625 nb_pixel_total : 1753 time to create 1 rle with old method : 0.0023005008697509766 length of segment : 68 time for calcul the mask position with numpy : 0.0014777183532714844 nb_pixel_total : 59440 time to create 1 rle with old method : 0.06853747367858887 length of segment : 313 time for calcul the mask position with numpy : 0.0019872188568115234 nb_pixel_total : 79212 time to create 1 rle with old method : 0.09194016456604004 length of segment : 272 time for calcul the mask position with numpy : 0.010318279266357422 nb_pixel_total : 380744 time to create 1 rle with new method : 0.06857848167419434 length of segment : 771 time for calcul the mask position with numpy : 0.0017426013946533203 nb_pixel_total : 52347 time to create 1 rle with old method : 0.061571359634399414 length of segment : 458 time for calcul the mask position with numpy : 0.0003361701965332031 nb_pixel_total : 6412 time to create 1 rle with old method : 0.007885217666625977 length of segment : 110 time for calcul the mask position with numpy : 0.004379987716674805 nb_pixel_total : 184505 time to create 1 rle with new method : 0.006381034851074219 length of segment : 452 time for calcul the mask position with numpy : 0.0012805461883544922 nb_pixel_total : 51806 time to create 1 rle with old method : 0.0633702278137207 length of segment : 305 time for calcul the mask position with numpy : 0.0007522106170654297 nb_pixel_total : 20027 time to create 1 rle with old method : 0.024487018585205078 length of segment : 209 time for calcul the mask position with numpy : 0.00010418891906738281 nb_pixel_total : 1762 time to create 1 rle with old method : 0.0031075477600097656 length of segment : 60 time for calcul the mask position with numpy : 0.0014426708221435547 nb_pixel_total : 62113 time to create 1 rle with old method : 0.09156346321105957 length of segment : 303 time for calcul the mask position with numpy : 0.0009253025054931641 nb_pixel_total : 8505 time to create 1 rle with old method : 0.0106201171875 length of segment : 216 time for calcul the mask position with numpy : 0.002042055130004883 nb_pixel_total : 63407 time to create 1 rle with old method : 0.074462890625 length of segment : 304 time for calcul the mask position with numpy : 0.0003077983856201172 nb_pixel_total : 5070 time to create 1 rle with old method : 0.006562232971191406 length of segment : 93 time for calcul the mask position with numpy : 0.005036592483520508 nb_pixel_total : 197965 time to create 1 rle with new method : 0.008381128311157227 length of segment : 460 time for calcul the mask position with numpy : 0.0003457069396972656 nb_pixel_total : 6353 time to create 1 rle with old method : 0.007736682891845703 length of segment : 109 time for calcul the mask position with numpy : 0.0017552375793457031 nb_pixel_total : 58180 time to create 1 rle with old method : 0.06482100486755371 length of segment : 392 time for calcul the mask position with numpy : 0.0010874271392822266 nb_pixel_total : 24549 time to create 1 rle with old method : 0.028888940811157227 length of segment : 143 time for calcul the mask position with numpy : 0.0008337497711181641 nb_pixel_total : 13763 time to create 1 rle with old method : 0.0158689022064209 length of segment : 188 time for calcul the mask position with numpy : 0.0013206005096435547 nb_pixel_total : 43678 time to create 1 rle with old method : 0.0544590950012207 length of segment : 325 time for calcul the mask position with numpy : 0.0004527568817138672 nb_pixel_total : 14313 time to create 1 rle with old method : 0.017222166061401367 length of segment : 160 time for calcul the mask position with numpy : 0.001135110855102539 nb_pixel_total : 34536 time to create 1 rle with old method : 0.038683414459228516 length of segment : 214 time for calcul the mask position with numpy : 0.004489898681640625 nb_pixel_total : 173181 time to create 1 rle with new method : 0.007591724395751953 length of segment : 747 time for calcul the mask position with numpy : 0.0036954879760742188 nb_pixel_total : 132418 time to create 1 rle with old method : 0.15171265602111816 length of segment : 618 time for calcul the mask position with numpy : 0.0011317729949951172 nb_pixel_total : 44166 time to create 1 rle with old method : 0.052129268646240234 length of segment : 379 time for calcul the mask position with numpy : 0.0016908645629882812 nb_pixel_total : 160 time to create 1 rle with old method : 0.0002913475036621094 length of segment : 43 time for calcul the mask position with numpy : 0.005957603454589844 nb_pixel_total : 221463 time to create 1 rle with new method : 0.009885549545288086 length of segment : 900 time for calcul the mask position with numpy : 0.004290342330932617 nb_pixel_total : 214869 time to create 1 rle with new method : 0.007588624954223633 length of segment : 587 time for calcul the mask position with numpy : 0.0005738735198974609 nb_pixel_total : 11439 time to create 1 rle with old method : 0.014699697494506836 length of segment : 116 time for calcul the mask position with numpy : 0.00676274299621582 nb_pixel_total : 84207 time to create 1 rle with old method : 0.13520455360412598 length of segment : 687 time for calcul the mask position with numpy : 0.00026535987854003906 nb_pixel_total : 2075 time to create 1 rle with old method : 0.002757549285888672 length of segment : 61 time for calcul the mask position with numpy : 0.007173299789428711 nb_pixel_total : 199927 time to create 1 rle with new method : 0.013077259063720703 length of segment : 645 time for calcul the mask position with numpy : 0.0007734298706054688 nb_pixel_total : 9828 time to create 1 rle with old method : 0.015117168426513672 length of segment : 212 time for calcul the mask position with numpy : 0.001482248306274414 nb_pixel_total : 9787 time to create 1 rle with old method : 0.01691722869873047 length of segment : 204 time for calcul the mask position with numpy : 0.0059392452239990234 nb_pixel_total : 87790 time to create 1 rle with old method : 0.10684084892272949 length of segment : 414 time for calcul the mask position with numpy : 0.0005197525024414062 nb_pixel_total : 10398 time to create 1 rle with old method : 0.012541770935058594 length of segment : 118 time for calcul the mask position with numpy : 0.0004315376281738281 nb_pixel_total : 7912 time to create 1 rle with old method : 0.011199951171875 length of segment : 77 time for calcul the mask position with numpy : 0.00038123130798339844 nb_pixel_total : 2039 time to create 1 rle with old method : 0.003059864044189453 length of segment : 66 time for calcul the mask position with numpy : 0.0019524097442626953 nb_pixel_total : 52897 time to create 1 rle with old method : 0.0659952163696289 length of segment : 414 time for calcul the mask position with numpy : 0.0008578300476074219 nb_pixel_total : 14901 time to create 1 rle with old method : 0.01773667335510254 length of segment : 180 time for calcul the mask position with numpy : 0.00012493133544921875 nb_pixel_total : 1876 time to create 1 rle with old method : 0.002393007278442383 length of segment : 74 time for calcul the mask position with numpy : 0.0036258697509765625 nb_pixel_total : 87799 time to create 1 rle with old method : 0.09912800788879395 length of segment : 401 time for calcul the mask position with numpy : 0.000972747802734375 nb_pixel_total : 15106 time to create 1 rle with old method : 0.019193172454833984 length of segment : 89 time for calcul the mask position with numpy : 0.00025916099548339844 nb_pixel_total : 2818 time to create 1 rle with old method : 0.003581523895263672 length of segment : 73 time for calcul the mask position with numpy : 0.003152608871459961 nb_pixel_total : 93140 time to create 1 rle with old method : 0.13031435012817383 length of segment : 368 time for calcul the mask position with numpy : 0.001773834228515625 nb_pixel_total : 92390 time to create 1 rle with old method : 0.10358023643493652 length of segment : 368 time for calcul the mask position with numpy : 0.0015347003936767578 nb_pixel_total : 28548 time to create 1 rle with old method : 0.035637617111206055 length of segment : 212 time for calcul the mask position with numpy : 0.002493619918823242 nb_pixel_total : 95334 time to create 1 rle with old method : 0.10942864418029785 length of segment : 405 time for calcul the mask position with numpy : 0.0002636909484863281 nb_pixel_total : 7104 time to create 1 rle with old method : 0.008620738983154297 length of segment : 99 time for calcul the mask position with numpy : 0.00257110595703125 nb_pixel_total : 50567 time to create 1 rle with old method : 0.05948185920715332 length of segment : 405 time for calcul the mask position with numpy : 0.0015914440155029297 nb_pixel_total : 47558 time to create 1 rle with old method : 0.05792951583862305 length of segment : 202 time for calcul the mask position with numpy : 0.00024890899658203125 nb_pixel_total : 6118 time to create 1 rle with old method : 0.007519245147705078 length of segment : 80 time for calcul the mask position with numpy : 0.01174783706665039 nb_pixel_total : 461065 time to create 1 rle with new method : 0.01787424087524414 length of segment : 724 time for calcul the mask position with numpy : 0.003313779830932617 nb_pixel_total : 45127 time to create 1 rle with old method : 0.07102465629577637 length of segment : 435 time for calcul the mask position with numpy : 0.00148773193359375 nb_pixel_total : 53831 time to create 1 rle with old method : 0.06225395202636719 length of segment : 223 time for calcul the mask position with numpy : 0.001039266586303711 nb_pixel_total : 53761 time to create 1 rle with old method : 0.06462788581848145 length of segment : 293 time for calcul the mask position with numpy : 0.004070281982421875 nb_pixel_total : 164992 time to create 1 rle with new method : 0.0057985782623291016 length of segment : 541 time for calcul the mask position with numpy : 0.0017011165618896484 nb_pixel_total : 47547 time to create 1 rle with old method : 0.05495405197143555 length of segment : 430 time for calcul the mask position with numpy : 0.0005257129669189453 nb_pixel_total : 6144 time to create 1 rle with old method : 0.007863998413085938 length of segment : 96 time for calcul the mask position with numpy : 0.001093149185180664 nb_pixel_total : 50937 time to create 1 rle with old method : 0.059229373931884766 length of segment : 207 time for calcul the mask position with numpy : 0.003916501998901367 nb_pixel_total : 123614 time to create 1 rle with old method : 0.14600062370300293 length of segment : 352 time for calcul the mask position with numpy : 0.0012097358703613281 nb_pixel_total : 56137 time to create 1 rle with old method : 0.06441807746887207 length of segment : 308 time for calcul the mask position with numpy : 0.0002651214599609375 nb_pixel_total : 4277 time to create 1 rle with old method : 0.00612640380859375 length of segment : 73 time for calcul the mask position with numpy : 0.0003924369812011719 nb_pixel_total : 10401 time to create 1 rle with old method : 0.015906333923339844 length of segment : 120 time for calcul the mask position with numpy : 0.0026938915252685547 nb_pixel_total : 101646 time to create 1 rle with old method : 0.1252422332763672 length of segment : 435 time for calcul the mask position with numpy : 0.00024080276489257812 nb_pixel_total : 4665 time to create 1 rle with old method : 0.007917404174804688 length of segment : 73 time for calcul the mask position with numpy : 0.005116939544677734 nb_pixel_total : 97718 time to create 1 rle with old method : 0.14841532707214355 length of segment : 492 time for calcul the mask position with numpy : 0.0009019374847412109 nb_pixel_total : 13515 time to create 1 rle with old method : 0.015915393829345703 length of segment : 206 time for calcul the mask position with numpy : 0.002910137176513672 nb_pixel_total : 68798 time to create 1 rle with old method : 0.07724189758300781 length of segment : 322 time for calcul the mask position with numpy : 0.0014348030090332031 nb_pixel_total : 25168 time to create 1 rle with old method : 0.0405733585357666 length of segment : 185 time for calcul the mask position with numpy : 0.0005443096160888672 nb_pixel_total : 8730 time to create 1 rle with old method : 0.010976314544677734 length of segment : 103 time for calcul the mask position with numpy : 0.0019254684448242188 nb_pixel_total : 48832 time to create 1 rle with old method : 0.05893087387084961 length of segment : 260 time for calcul the mask position with numpy : 0.0012710094451904297 nb_pixel_total : 26504 time to create 1 rle with old method : 0.03137636184692383 length of segment : 340 time for calcul the mask position with numpy : 0.0007948875427246094 nb_pixel_total : 14323 time to create 1 rle with old method : 0.01610875129699707 length of segment : 211 time for calcul the mask position with numpy : 0.008724212646484375 nb_pixel_total : 268395 time to create 1 rle with new method : 0.012688636779785156 length of segment : 691 time for calcul the mask position with numpy : 0.0006992816925048828 nb_pixel_total : 9778 time to create 1 rle with old method : 0.01613450050354004 length of segment : 147 time for calcul the mask position with numpy : 0.001905202865600586 nb_pixel_total : 31379 time to create 1 rle with old method : 0.04119253158569336 length of segment : 252 time for calcul the mask position with numpy : 0.0003724098205566406 nb_pixel_total : 5708 time to create 1 rle with old method : 0.0068356990814208984 length of segment : 106 time for calcul the mask position with numpy : 0.002640247344970703 nb_pixel_total : 176250 time to create 1 rle with new method : 0.006849050521850586 length of segment : 710 time for calcul the mask position with numpy : 0.012389183044433594 nb_pixel_total : 227407 time to create 1 rle with new method : 0.019712209701538086 length of segment : 720 time for calcul the mask position with numpy : 0.0016944408416748047 nb_pixel_total : 109202 time to create 1 rle with old method : 0.1258707046508789 length of segment : 241 time for calcul the mask position with numpy : 0.0006237030029296875 nb_pixel_total : 23621 time to create 1 rle with old method : 0.026801347732543945 length of segment : 327 time for calcul the mask position with numpy : 0.0005233287811279297 nb_pixel_total : 9213 time to create 1 rle with old method : 0.01076197624206543 length of segment : 131 time for calcul the mask position with numpy : 0.00015926361083984375 nb_pixel_total : 3690 time to create 1 rle with old method : 0.004503488540649414 length of segment : 109 time for calcul the mask position with numpy : 0.0005252361297607422 nb_pixel_total : 10937 time to create 1 rle with old method : 0.01250910758972168 length of segment : 91 time for calcul the mask position with numpy : 0.0019309520721435547 nb_pixel_total : 36581 time to create 1 rle with old method : 0.06941533088684082 length of segment : 333 time for calcul the mask position with numpy : 0.0025641918182373047 nb_pixel_total : 66596 time to create 1 rle with old method : 0.07660341262817383 length of segment : 477 time for calcul the mask position with numpy : 0.0006947517395019531 nb_pixel_total : 10510 time to create 1 rle with old method : 0.012255430221557617 length of segment : 277 time for calcul the mask position with numpy : 0.00031113624572753906 nb_pixel_total : 13251 time to create 1 rle with old method : 0.019170761108398438 length of segment : 108 time for calcul the mask position with numpy : 0.0001690387725830078 nb_pixel_total : 907 time to create 1 rle with old method : 0.0012056827545166016 length of segment : 38 time for calcul the mask position with numpy : 0.0049741268157958984 nb_pixel_total : 80974 time to create 1 rle with old method : 0.158158540725708 length of segment : 638 time for calcul the mask position with numpy : 0.0005545616149902344 nb_pixel_total : 7448 time to create 1 rle with old method : 0.009155035018920898 length of segment : 143 time for calcul the mask position with numpy : 0.009317636489868164 nb_pixel_total : 148561 time to create 1 rle with old method : 0.16698241233825684 length of segment : 741 time for calcul the mask position with numpy : 0.0009555816650390625 nb_pixel_total : 23609 time to create 1 rle with old method : 0.03812742233276367 length of segment : 167 time for calcul the mask position with numpy : 0.00029754638671875 nb_pixel_total : 2312 time to create 1 rle with old method : 0.0028810501098632812 length of segment : 89 time for calcul the mask position with numpy : 0.0022728443145751953 nb_pixel_total : 73784 time to create 1 rle with old method : 0.08745121955871582 length of segment : 338 time for calcul the mask position with numpy : 0.000244140625 nb_pixel_total : 3568 time to create 1 rle with old method : 0.004532575607299805 length of segment : 72 time for calcul the mask position with numpy : 0.002225637435913086 nb_pixel_total : 49418 time to create 1 rle with old method : 0.05656766891479492 length of segment : 444 time for calcul the mask position with numpy : 0.0017528533935546875 nb_pixel_total : 37691 time to create 1 rle with old method : 0.04800009727478027 length of segment : 388 time for calcul the mask position with numpy : 0.0011014938354492188 nb_pixel_total : 22341 time to create 1 rle with old method : 0.0289003849029541 length of segment : 193 time for calcul the mask position with numpy : 0.0035123825073242188 nb_pixel_total : 75833 time to create 1 rle with old method : 0.09599065780639648 length of segment : 532 time for calcul the mask position with numpy : 0.0022788047790527344 nb_pixel_total : 45786 time to create 1 rle with old method : 0.051288604736328125 length of segment : 333 time for calcul the mask position with numpy : 0.002875804901123047 nb_pixel_total : 17858 time to create 1 rle with old method : 0.02090620994567871 length of segment : 168 time for calcul the mask position with numpy : 0.0005998611450195312 nb_pixel_total : 11443 time to create 1 rle with old method : 0.013128042221069336 length of segment : 162 time for calcul the mask position with numpy : 0.006362199783325195 nb_pixel_total : 132061 time to create 1 rle with old method : 0.15565776824951172 length of segment : 547 time for calcul the mask position with numpy : 0.0021271705627441406 nb_pixel_total : 53950 time to create 1 rle with old method : 0.06171536445617676 length of segment : 512 time for calcul the mask position with numpy : 0.0050580501556396484 nb_pixel_total : 197645 time to create 1 rle with new method : 0.008527278900146484 length of segment : 624 time for calcul the mask position with numpy : 0.0017027854919433594 nb_pixel_total : 41068 time to create 1 rle with old method : 0.04719042778015137 length of segment : 442 time for calcul the mask position with numpy : 0.00283050537109375 nb_pixel_total : 61254 time to create 1 rle with old method : 0.0692284107208252 length of segment : 292 time for calcul the mask position with numpy : 0.002791166305541992 nb_pixel_total : 37038 time to create 1 rle with old method : 0.0456087589263916 length of segment : 290 time for calcul the mask position with numpy : 0.0026092529296875 nb_pixel_total : 62277 time to create 1 rle with old method : 0.06920385360717773 length of segment : 597 time for calcul the mask position with numpy : 0.006721019744873047 nb_pixel_total : 107134 time to create 1 rle with old method : 0.12057852745056152 length of segment : 476 time for calcul the mask position with numpy : 0.0018672943115234375 nb_pixel_total : 45938 time to create 1 rle with old method : 0.055185556411743164 length of segment : 432 time for calcul the mask position with numpy : 0.0002532005310058594 nb_pixel_total : 1493 time to create 1 rle with old method : 0.0020263195037841797 length of segment : 75 time for calcul the mask position with numpy : 0.001127481460571289 nb_pixel_total : 29161 time to create 1 rle with old method : 0.035500526428222656 length of segment : 248 time for calcul the mask position with numpy : 0.00016498565673828125 nb_pixel_total : 816 time to create 1 rle with old method : 0.00119781494140625 length of segment : 23 time for calcul the mask position with numpy : 0.0025081634521484375 nb_pixel_total : 26023 time to create 1 rle with old method : 0.03239631652832031 length of segment : 158 time for calcul the mask position with numpy : 0.001312255859375 nb_pixel_total : 28455 time to create 1 rle with old method : 0.03369545936584473 length of segment : 231 time for calcul the mask position with numpy : 0.0002968311309814453 nb_pixel_total : 3996 time to create 1 rle with old method : 0.006142139434814453 length of segment : 90 time for calcul the mask position with numpy : 0.003980875015258789 nb_pixel_total : 76247 time to create 1 rle with old method : 0.1012418270111084 length of segment : 297 time for calcul the mask position with numpy : 0.0006999969482421875 nb_pixel_total : 24684 time to create 1 rle with old method : 0.029884815216064453 length of segment : 212 time for calcul the mask position with numpy : 0.0014674663543701172 nb_pixel_total : 55902 time to create 1 rle with old method : 0.06684494018554688 length of segment : 275 time for calcul the mask position with numpy : 0.00038051605224609375 nb_pixel_total : 6176 time to create 1 rle with old method : 0.007663249969482422 length of segment : 112 time for calcul the mask position with numpy : 0.0076143741607666016 nb_pixel_total : 165555 time to create 1 rle with new method : 0.011837482452392578 length of segment : 788 time for calcul the mask position with numpy : 0.01821136474609375 nb_pixel_total : 462559 time to create 1 rle with new method : 0.1657552719116211 length of segment : 903 time for calcul the mask position with numpy : 0.0027403831481933594 nb_pixel_total : 71729 time to create 1 rle with old method : 0.08324980735778809 length of segment : 353 time for calcul the mask position with numpy : 0.00031828880310058594 nb_pixel_total : 2629 time to create 1 rle with old method : 0.0035505294799804688 length of segment : 40 time for calcul the mask position with numpy : 0.0026171207427978516 nb_pixel_total : 52525 time to create 1 rle with old method : 0.06393957138061523 length of segment : 327 time for calcul the mask position with numpy : 0.000637054443359375 nb_pixel_total : 6350 time to create 1 rle with old method : 0.02664923667907715 length of segment : 100 time for calcul the mask position with numpy : 0.010729074478149414 nb_pixel_total : 129547 time to create 1 rle with old method : 0.16760730743408203 length of segment : 794 time for calcul the mask position with numpy : 0.0010097026824951172 nb_pixel_total : 37900 time to create 1 rle with old method : 0.0449373722076416 length of segment : 212 time for calcul the mask position with numpy : 0.0020570755004882812 nb_pixel_total : 51476 time to create 1 rle with old method : 0.06332778930664062 length of segment : 289 time for calcul the mask position with numpy : 0.0023772716522216797 nb_pixel_total : 27935 time to create 1 rle with old method : 0.0319666862487793 length of segment : 577 time for calcul the mask position with numpy : 0.0004966259002685547 nb_pixel_total : 8404 time to create 1 rle with old method : 0.010197877883911133 length of segment : 142 time for calcul the mask position with numpy : 0.0005030632019042969 nb_pixel_total : 8358 time to create 1 rle with old method : 0.01023554801940918 length of segment : 126 time for calcul the mask position with numpy : 0.005223512649536133 nb_pixel_total : 103163 time to create 1 rle with old method : 0.11447620391845703 length of segment : 609 time for calcul the mask position with numpy : 0.004633903503417969 nb_pixel_total : 99155 time to create 1 rle with old method : 0.11807107925415039 length of segment : 485 time for calcul the mask position with numpy : 0.0052525997161865234 nb_pixel_total : 108512 time to create 1 rle with old method : 0.14474749565124512 length of segment : 572 time for calcul the mask position with numpy : 0.0017137527465820312 nb_pixel_total : 50893 time to create 1 rle with old method : 0.05956530570983887 length of segment : 189 time for calcul the mask position with numpy : 0.0012996196746826172 nb_pixel_total : 20991 time to create 1 rle with old method : 0.02410745620727539 length of segment : 337 time for calcul the mask position with numpy : 0.0012097358703613281 nb_pixel_total : 24939 time to create 1 rle with old method : 0.02767205238342285 length of segment : 317 time for calcul the mask position with numpy : 0.0001926422119140625 nb_pixel_total : 1741 time to create 1 rle with old method : 0.0022666454315185547 length of segment : 47 time for calcul the mask position with numpy : 0.0008778572082519531 nb_pixel_total : 22729 time to create 1 rle with old method : 0.02578592300415039 length of segment : 186 time for calcul the mask position with numpy : 0.0033330917358398438 nb_pixel_total : 90963 time to create 1 rle with old method : 0.09962940216064453 length of segment : 454 time for calcul the mask position with numpy : 0.002775430679321289 nb_pixel_total : 48685 time to create 1 rle with old method : 0.05633687973022461 length of segment : 278 time for calcul the mask position with numpy : 0.0024688243865966797 nb_pixel_total : 128431 time to create 1 rle with old method : 0.1458580493927002 length of segment : 402 time for calcul the mask position with numpy : 0.0012061595916748047 nb_pixel_total : 33342 time to create 1 rle with old method : 0.03968667984008789 length of segment : 221 time for calcul the mask position with numpy : 0.0008983612060546875 nb_pixel_total : 38317 time to create 1 rle with old method : 0.04419350624084473 length of segment : 288 time for calcul the mask position with numpy : 0.0005397796630859375 nb_pixel_total : 25948 time to create 1 rle with old method : 0.0311892032623291 length of segment : 226 time for calcul the mask position with numpy : 0.00048470497131347656 nb_pixel_total : 3985 time to create 1 rle with old method : 0.0051021575927734375 length of segment : 92 time for calcul the mask position with numpy : 0.003064393997192383 nb_pixel_total : 140025 time to create 1 rle with old method : 0.15248680114746094 length of segment : 572 time for calcul the mask position with numpy : 0.0009293556213378906 nb_pixel_total : 46264 time to create 1 rle with old method : 0.05477166175842285 length of segment : 304 time spent for convertir_results : 22.26226305961609 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 229 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 65529 save missing photos in datou_result : time spend for datou_step_exec : 79.9966824054718 time spend to save output : 16.513669729232788 total time spend for step 1 : 96.51035213470459 step2:crop_condition Tue Mar 11 15:22:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 229 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 41 About to insert : list_path_to_insert length 41 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 58 About to insert : list_path_to_insert length 58 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 141 /-3705556289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556296Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621807Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556346Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556345Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621813Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556416Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621812Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556490Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621808Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556321Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556310Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556341Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556347Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556349Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556355Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621814Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556397Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556361Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705621810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3705556454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023136', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023075', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022775', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022749', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022747', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022739', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022531', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022217', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022175', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022101', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 433 time used for this insertion : 0.04989051818847656 save_final save missing photos in datou_result : time spend for datou_step_exec : 59.2225604057312 time spend to save output : 0.0539553165435791 total time spend for step 2 : 59.27651572227478 step3:thcl Tue Mar 11 15:23:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.006935596466064453 time to convert the images to numpy array : 0.23048639297485352 time to import caffe and check if the image exist : 0.019066810607910156 time to convert the images to numpy array : 0.27504706382751465 time to import caffe and check if the image exist : 0.010302305221557617 time to convert the images to numpy array : 0.28452324867248535 time to import caffe and check if the image exist : 0.010812759399414062 time to convert the images to numpy array : 0.2848176956176758 time to import caffe and check if the image exist : 0.012901067733764648 time to convert the images to numpy array : 0.2863492965698242 time to import caffe and check if the image exist : 0.012499094009399414 time to convert the images to numpy array : 0.29740238189697266 time to import caffe and check if the image exist : 0.014862298965454102 time to convert the images to numpy array : 0.3075101375579834 time to import caffe and check if the image exist : 0.012703657150268555 time to convert the images to numpy array : 0.34795069694519043 time to import caffe and check if the image exist : 0.01997828483581543 time to convert the images to numpy array : 0.3537757396697998 time to import caffe and check if the image exist : 0.019320964813232422 time to convert the images to numpy array : 0.5398561954498291 total time to convert the images to numpy array : 0.924746036529541 list photo_ids error: [] list photo_ids correct : [-3705556329, -3705556335, -3705556378, -3705556473, -3705556451, -3705556454, -3705621824, -3705556466, -3705556287, -3705556291, -3705556294, -3705556316, -3705621806, -3705556300, -3705556361, -3705556363, -3705556377, -3705556442, -3705556482, -3705621810, -3705556323, -3705556382, -3705556393, -3705556398, -3705556390, -3705556422, -3705556423, -3705556411, -3705556410, -3705556426, -3705556421, -3705556425, -3705621818, -3705556416, -3705556441, -3705556438, -3705556445, -3705556443, -3705556444, -3705556450, -3705556472, -3705556486, -3705556468, -3705556470, -3705556478, -3705556476, -3705556477, -3705556322, -3705556340, -3705621812, -3705556336, -3705556453, -3705556458, -3705556488, -3705556480, -3705621805, -3705556318, -3705556321, -3705556303, -3705556327, -3705556315, -3705621809, -3705621811, -3705556310, -3705556332, -3705556341, -3705556334, -3705556337, -3705556353, -3705556490, -3705556475, -3705556326, -3705556311, -3705556320, -3705621808, -3705556324, -3705556319, -3705556358, -3705556387, -3705556419, -3705556464, -3705556412, -3705556434, -3705556415, -3705621820, -3705556459, -3705556462, -3705556455, -3705556440, -3705556448, -3705556447, -3705556493, -3705556491, -3705556483, -3705621823, -3705621822, -3705556289, -3705556308, -3705556299, -3705556296, -3705556298, -3705556302, -3705621807, -3705556328, -3705556346, -3705556345, -3705556359, -3705556360, -3705621813, -3705556375, -3705556372, -3705556347, -3705556357, -3705556344, -3705556354, -3705556349, -3705556355, -3705556364, -3705556366, -3705556370, -3705621814, -3705556365, -3705556367, -3705556381, -3705556384, -3705556388, -3705556374, -3705556371, -3705556402, -3705556392, -3705556400, -3705556396, -3705556397, -3705556403, -3705556404, -3705556399, -3705621821, -3705556430, -3705556417, -3705621819, -3705621815] number of photos to traite : 141 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10374 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.4913074970245361 time used to do the prediction : 0.2887413501739502 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10372 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.5324547290802002 time used to do the prediction : 0.2644004821777344 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 3.0040740966796875e-05 save missing photos in datou_result : time spend for datou_step_exec : 11.78910779953003 time spend to save output : 0.0012607574462890625 total time spend for step 3 : 11.790368556976318 step4:merge_mask_thcl_custom Tue Mar 11 15:23:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 229 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 10 /1343023136Didn't retrieve data .Didn't retrieve data . /1343023075Didn't retrieve data .Didn't retrieve data . /1343022775Didn't retrieve data .Didn't retrieve data . /1343022749Didn't retrieve data .Didn't retrieve data . /1343022747Didn't retrieve data .Didn't retrieve data . /1343022739Didn't retrieve data .Didn't retrieve data . /1343022531Didn't retrieve data .Didn't retrieve data . /1343022217Didn't retrieve data .Didn't retrieve data . /1343022175Didn't retrieve data .Didn't retrieve data . /1343022101Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023136', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023075', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022775', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022749', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022747', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022739', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022531', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022217', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022175', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022101', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.016737699508666992 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.43436717987060547 time spend to save output : 0.017264366149902344 total time spend for step 4 : 0.4516315460205078 step5:rle_unique_nms_with_priority Tue Mar 11 15:23:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 10.001254558563232 time for calcul the mask position with numpy : 0.3080167770385742 nb_pixel_total : 2876587 time to create 1 rle with new method : 0.8752396106719971 time for calcul the mask position with numpy : 0.033463239669799805 nb_pixel_total : 2907 time to create 1 rle with old method : 0.003118276596069336 time for calcul the mask position with numpy : 0.025089502334594727 nb_pixel_total : 15218 time to create 1 rle with old method : 0.01639103889465332 time for calcul the mask position with numpy : 0.021265268325805664 nb_pixel_total : 14141 time to create 1 rle with old method : 0.015003681182861328 time for calcul the mask position with numpy : 0.02178478240966797 nb_pixel_total : 38536 time to create 1 rle with old method : 0.04234504699707031 time for calcul the mask position with numpy : 0.023847579956054688 nb_pixel_total : 137397 time to create 1 rle with old method : 0.15196466445922852 time for calcul the mask position with numpy : 0.021478652954101562 nb_pixel_total : 8767 time to create 1 rle with old method : 0.009593963623046875 time for calcul the mask position with numpy : 0.19577908515930176 nb_pixel_total : 3864996 time to create 1 rle with new method : 1.2079389095306396 time for calcul the mask position with numpy : 0.021581411361694336 nb_pixel_total : 91691 time to create 1 rle with old method : 0.09883856773376465 create new chi : 3.152162790298462 time to delete rle : 0.013476133346557617 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 11022 TO DO : save crop sub photo not yet done ! save time : 14.31604790687561 nb_obj : 33 nb_hashtags : 7 time to prepare the origin masks : 11.125705480575562 time for calcul the mask position with numpy : 0.7032663822174072 nb_pixel_total : 5619417 time to create 1 rle with new method : 0.948279857635498 time for calcul the mask position with numpy : 0.031600236892700195 nb_pixel_total : 151311 time to create 1 rle with new method : 1.5357508659362793 time for calcul the mask position with numpy : 0.03240251541137695 nb_pixel_total : 43034 time to create 1 rle with old method : 0.07236742973327637 time for calcul the mask position with numpy : 0.031738996505737305 nb_pixel_total : 19323 time to create 1 rle with old method : 0.023523569107055664 time for calcul the mask position with numpy : 0.03688669204711914 nb_pixel_total : 33434 time to create 1 rle with old method : 0.039166927337646484 time for calcul the mask position with numpy : 0.03178668022155762 nb_pixel_total : 16826 time to create 1 rle with old method : 0.021333932876586914 time for calcul the mask position with numpy : 0.031158447265625 nb_pixel_total : 1953 time to create 1 rle with old method : 0.0023925304412841797 time for calcul the mask position with numpy : 0.03110218048095703 nb_pixel_total : 28722 time to create 1 rle with old method : 0.033780574798583984 time for calcul the mask position with numpy : 0.03377938270568848 nb_pixel_total : 38364 time to create 1 rle with old method : 0.04850435256958008 time for calcul the mask position with numpy : 0.032245635986328125 nb_pixel_total : 81712 time to create 1 rle with old method : 0.09498143196105957 time for calcul the mask position with numpy : 0.031304359436035156 nb_pixel_total : 1769 time to create 1 rle with old method : 0.0029141902923583984 time for calcul the mask position with numpy : 0.033000946044921875 nb_pixel_total : 1785 time to create 1 rle with old method : 0.0029604434967041016 time for calcul the mask position with numpy : 0.03302478790283203 nb_pixel_total : 47568 time to create 1 rle with old method : 0.055764198303222656 time for calcul the mask position with numpy : 0.031055688858032227 nb_pixel_total : 129474 time to create 1 rle with old method : 0.15605688095092773 time for calcul the mask position with numpy : 0.02968621253967285 nb_pixel_total : 94589 time to create 1 rle with old method : 0.10828232765197754 time for calcul the mask position with numpy : 0.031922340393066406 nb_pixel_total : 45994 time to create 1 rle with old method : 0.052085161209106445 time for calcul the mask position with numpy : 0.029089927673339844 nb_pixel_total : 22800 time to create 1 rle with old method : 0.026282548904418945 time for calcul the mask position with numpy : 0.030798912048339844 nb_pixel_total : 102203 time to create 1 rle with old method : 0.11344695091247559 time for calcul the mask position with numpy : 0.03424811363220215 nb_pixel_total : 28458 time to create 1 rle with old method : 0.04421806335449219 time for calcul the mask position with numpy : 0.03444719314575195 nb_pixel_total : 26036 time to create 1 rle with old method : 0.03290915489196777 time for calcul the mask position with numpy : 0.0402982234954834 nb_pixel_total : 168324 time to create 1 rle with new method : 1.4277031421661377 time for calcul the mask position with numpy : 0.03564953804016113 nb_pixel_total : 67434 time to create 1 rle with old method : 0.08542275428771973 time for calcul the mask position with numpy : 0.04005122184753418 nb_pixel_total : 31083 time to create 1 rle with old method : 0.0374300479888916 time for calcul the mask position with numpy : 0.0335087776184082 nb_pixel_total : 25082 time to create 1 rle with old method : 0.030122041702270508 time for calcul the mask position with numpy : 0.03612208366394043 nb_pixel_total : 18945 time to create 1 rle with old method : 0.02829289436340332 time for calcul the mask position with numpy : 0.03285360336303711 nb_pixel_total : 2408 time to create 1 rle with old method : 0.0030906200408935547 time for calcul the mask position with numpy : 0.03098750114440918 nb_pixel_total : 11985 time to create 1 rle with old method : 0.016380786895751953 time for calcul the mask position with numpy : 0.031042814254760742 nb_pixel_total : 50898 time to create 1 rle with old method : 0.06281590461730957 time for calcul the mask position with numpy : 0.03202533721923828 nb_pixel_total : 36698 time to create 1 rle with old method : 0.0406038761138916 time for calcul the mask position with numpy : 0.03304576873779297 nb_pixel_total : 12581 time to create 1 rle with old method : 0.015432119369506836 time for calcul the mask position with numpy : 0.03368949890136719 nb_pixel_total : 20731 time to create 1 rle with old method : 0.025949716567993164 time for calcul the mask position with numpy : 0.03594160079956055 nb_pixel_total : 4682 time to create 1 rle with old method : 0.005495548248291016 time for calcul the mask position with numpy : 0.030562400817871094 nb_pixel_total : 17026 time to create 1 rle with old method : 0.02355813980102539 time for calcul the mask position with numpy : 0.030868053436279297 nb_pixel_total : 47591 time to create 1 rle with old method : 0.07251596450805664 create new chi : 7.184748888015747 time to delete rle : 0.002462148666381836 batch 1 Loaded 40 chid ids of type : 3726 Number RLEs to save : 18990 TO DO : save crop sub photo not yet done ! save time : 17.830575704574585 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 11.420913696289062 time for calcul the mask position with numpy : 0.6233179569244385 nb_pixel_total : 6541185 time to create 1 rle with new method : 0.6563000679016113 time for calcul the mask position with numpy : 0.035318851470947266 nb_pixel_total : 11534 time to create 1 rle with old method : 0.018528461456298828 time for calcul the mask position with numpy : 0.024034738540649414 nb_pixel_total : 67315 time to create 1 rle with old method : 0.08048653602600098 time for calcul the mask position with numpy : 0.021389484405517578 nb_pixel_total : 30298 time to create 1 rle with old method : 0.03912210464477539 time for calcul the mask position with numpy : 0.02395915985107422 nb_pixel_total : 118430 time to create 1 rle with old method : 0.1335747241973877 time for calcul the mask position with numpy : 0.02665853500366211 nb_pixel_total : 103494 time to create 1 rle with old method : 0.12487006187438965 time for calcul the mask position with numpy : 0.023569345474243164 nb_pixel_total : 41675 time to create 1 rle with old method : 0.04963541030883789 time for calcul the mask position with numpy : 0.0233461856842041 nb_pixel_total : 14047 time to create 1 rle with old method : 0.017101049423217773 time for calcul the mask position with numpy : 0.03332662582397461 nb_pixel_total : 707 time to create 1 rle with old method : 0.0010972023010253906 time for calcul the mask position with numpy : 0.03690910339355469 nb_pixel_total : 4533 time to create 1 rle with old method : 0.005708932876586914 time for calcul the mask position with numpy : 0.0372929573059082 nb_pixel_total : 50125 time to create 1 rle with old method : 0.05470156669616699 time for calcul the mask position with numpy : 0.03684878349304199 nb_pixel_total : 61785 time to create 1 rle with old method : 0.0663151741027832 time for calcul the mask position with numpy : 0.03641080856323242 nb_pixel_total : 5112 time to create 1 rle with old method : 0.0055408477783203125 create new chi : 2.2714836597442627 time to delete rle : 0.0008792877197265625 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 8549 TO DO : save crop sub photo not yet done ! save time : 0.8488016128540039 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 16.960971355438232 time for calcul the mask position with numpy : 0.3677382469177246 nb_pixel_total : 5904782 time to create 1 rle with new method : 0.4894444942474365 time for calcul the mask position with numpy : 0.042374372482299805 nb_pixel_total : 197543 time to create 1 rle with new method : 0.8591878414154053 time for calcul the mask position with numpy : 0.04093647003173828 nb_pixel_total : 59273 time to create 1 rle with old method : 0.07717752456665039 time for calcul the mask position with numpy : 0.036698102951049805 nb_pixel_total : 6341 time to create 1 rle with old method : 0.006930351257324219 time for calcul the mask position with numpy : 0.038275718688964844 nb_pixel_total : 5053 time to create 1 rle with old method : 0.005802631378173828 time for calcul the mask position with numpy : 0.03518366813659668 nb_pixel_total : 1759 time to create 1 rle with old method : 0.001959085464477539 time for calcul the mask position with numpy : 0.03572535514831543 nb_pixel_total : 78843 time to create 1 rle with old method : 0.10196161270141602 time for calcul the mask position with numpy : 0.04146265983581543 nb_pixel_total : 19923 time to create 1 rle with old method : 0.022486448287963867 time for calcul the mask position with numpy : 0.03504037857055664 nb_pixel_total : 51200 time to create 1 rle with old method : 0.05830216407775879 time for calcul the mask position with numpy : 0.03862881660461426 nb_pixel_total : 52183 time to create 1 rle with old method : 0.05945110321044922 time for calcul the mask position with numpy : 0.03520965576171875 nb_pixel_total : 3293 time to create 1 rle with old method : 0.006227016448974609 time for calcul the mask position with numpy : 0.04315185546875 nb_pixel_total : 380218 time to create 1 rle with new method : 0.7754724025726318 time for calcul the mask position with numpy : 0.030298948287963867 nb_pixel_total : 184707 time to create 1 rle with new method : 0.5346450805664062 time for calcul the mask position with numpy : 0.035399436950683594 nb_pixel_total : 6395 time to create 1 rle with old method : 0.007172822952270508 time for calcul the mask position with numpy : 0.0340421199798584 nb_pixel_total : 61946 time to create 1 rle with old method : 0.06552791595458984 time for calcul the mask position with numpy : 0.03451132774353027 nb_pixel_total : 1745 time to create 1 rle with old method : 0.001962900161743164 time for calcul the mask position with numpy : 0.033734798431396484 nb_pixel_total : 8465 time to create 1 rle with old method : 0.009203672409057617 time for calcul the mask position with numpy : 0.03379940986633301 nb_pixel_total : 26571 time to create 1 rle with old method : 0.03312087059020996 create new chi : 4.237331867218018 time to delete rle : 0.0011720657348632812 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 11430 TO DO : save crop sub photo not yet done ! save time : 5.334714412689209 nb_obj : 11 nb_hashtags : 5 time to prepare the origin masks : 14.041740655899048 time for calcul the mask position with numpy : 1.189295768737793 nb_pixel_total : 6104904 time to create 1 rle with new method : 1.504399061203003 time for calcul the mask position with numpy : 0.02153301239013672 nb_pixel_total : 34365 time to create 1 rle with old method : 0.044675350189208984 time for calcul the mask position with numpy : 0.02368474006652832 nb_pixel_total : 13728 time to create 1 rle with old method : 0.01757526397705078 time for calcul the mask position with numpy : 0.02137613296508789 nb_pixel_total : 15731 time to create 1 rle with old method : 0.01758432388305664 time for calcul the mask position with numpy : 0.02186441421508789 nb_pixel_total : 43565 time to create 1 rle with old method : 0.04902362823486328 time for calcul the mask position with numpy : 0.024067163467407227 nb_pixel_total : 24476 time to create 1 rle with old method : 0.02864527702331543 time for calcul the mask position with numpy : 0.02316117286682129 nb_pixel_total : 214717 time to create 1 rle with new method : 0.8667597770690918 time for calcul the mask position with numpy : 0.028401851654052734 nb_pixel_total : 58114 time to create 1 rle with old method : 0.06914424896240234 time for calcul the mask position with numpy : 0.026434898376464844 nb_pixel_total : 132320 time to create 1 rle with old method : 0.15564942359924316 time for calcul the mask position with numpy : 0.023177385330200195 nb_pixel_total : 14204 time to create 1 rle with old method : 0.016772985458374023 time for calcul the mask position with numpy : 0.02587747573852539 nb_pixel_total : 221245 time to create 1 rle with new method : 0.9499979019165039 time for calcul the mask position with numpy : 0.022543907165527344 nb_pixel_total : 172871 time to create 1 rle with new method : 0.7050971984863281 create new chi : 6.002817630767822 time to delete rle : 0.001653909683227539 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 11221 TO DO : save crop sub photo not yet done ! save time : 11.459312200546265 nb_obj : 17 nb_hashtags : 7 time to prepare the origin masks : 13.044806003570557 time for calcul the mask position with numpy : 0.4204721450805664 nb_pixel_total : 6386581 time to create 1 rle with new method : 0.6686854362487793 time for calcul the mask position with numpy : 0.022313833236694336 nb_pixel_total : 989 time to create 1 rle with old method : 0.001519918441772461 time for calcul the mask position with numpy : 0.023000717163085938 nb_pixel_total : 11298 time to create 1 rle with old method : 0.01401066780090332 time for calcul the mask position with numpy : 0.022345304489135742 nb_pixel_total : 92978 time to create 1 rle with old method : 0.09767937660217285 time for calcul the mask position with numpy : 0.020328044891357422 nb_pixel_total : 14663 time to create 1 rle with old method : 0.015577077865600586 time for calcul the mask position with numpy : 0.025159120559692383 nb_pixel_total : 72936 time to create 1 rle with old method : 0.07959103584289551 time for calcul the mask position with numpy : 0.023317575454711914 nb_pixel_total : 10387 time to create 1 rle with old method : 0.012601375579833984 time for calcul the mask position with numpy : 0.022909879684448242 nb_pixel_total : 2809 time to create 1 rle with old method : 0.003762483596801758 time for calcul the mask position with numpy : 0.023430585861206055 nb_pixel_total : 53103 time to create 1 rle with old method : 0.057860612869262695 time for calcul the mask position with numpy : 0.02195262908935547 nb_pixel_total : 1853 time to create 1 rle with old method : 0.002108335494995117 time for calcul the mask position with numpy : 0.02266216278076172 nb_pixel_total : 2016 time to create 1 rle with old method : 0.0023255348205566406 time for calcul the mask position with numpy : 0.022780895233154297 nb_pixel_total : 7887 time to create 1 rle with old method : 0.008716583251953125 time for calcul the mask position with numpy : 0.023164033889770508 nb_pixel_total : 83952 time to create 1 rle with old method : 0.09180569648742676 time for calcul the mask position with numpy : 0.023415565490722656 nb_pixel_total : 9799 time to create 1 rle with old method : 0.010853052139282227 time for calcul the mask position with numpy : 0.02369236946105957 nb_pixel_total : 9629 time to create 1 rle with old method : 0.012647867202758789 time for calcul the mask position with numpy : 0.02290511131286621 nb_pixel_total : 87459 time to create 1 rle with old method : 0.09527826309204102 time for calcul the mask position with numpy : 0.022596359252929688 nb_pixel_total : 2051 time to create 1 rle with old method : 0.0024161338806152344 time for calcul the mask position with numpy : 0.025609731674194336 nb_pixel_total : 199850 time to create 1 rle with new method : 0.8911631107330322 create new chi : 2.9958136081695557 time to delete rle : 0.0011620521545410156 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 10315 TO DO : save crop sub photo not yet done ! save time : 8.533790826797485 nb_obj : 17 nb_hashtags : 6 time to prepare the origin masks : 12.12773084640503 time for calcul the mask position with numpy : 1.6231944561004639 nb_pixel_total : 5874794 time to create 1 rle with new method : 1.1834442615509033 time for calcul the mask position with numpy : 0.022594690322875977 nb_pixel_total : 44996 time to create 1 rle with old method : 0.04943656921386719 time for calcul the mask position with numpy : 0.022479772567749023 nb_pixel_total : 123399 time to create 1 rle with old method : 0.1347513198852539 time for calcul the mask position with numpy : 0.022174596786499023 nb_pixel_total : 3556 time to create 1 rle with old method : 0.0045015811920166016 time for calcul the mask position with numpy : 0.022541046142578125 nb_pixel_total : 4590 time to create 1 rle with old method : 0.0052242279052734375 time for calcul the mask position with numpy : 0.023975133895874023 nb_pixel_total : 4145 time to create 1 rle with old method : 0.004912614822387695 time for calcul the mask position with numpy : 0.023834943771362305 nb_pixel_total : 47366 time to create 1 rle with old method : 0.05341005325317383 time for calcul the mask position with numpy : 0.024749279022216797 nb_pixel_total : 164669 time to create 1 rle with new method : 1.637800693511963 time for calcul the mask position with numpy : 0.026004791259765625 nb_pixel_total : 6129 time to create 1 rle with old method : 0.006779670715332031 time for calcul the mask position with numpy : 0.027687788009643555 nb_pixel_total : 460218 time to create 1 rle with new method : 1.0943396091461182 time for calcul the mask position with numpy : 0.03692913055419922 nb_pixel_total : 5917 time to create 1 rle with old method : 0.006987810134887695 time for calcul the mask position with numpy : 0.03603816032409668 nb_pixel_total : 47373 time to create 1 rle with old method : 0.0550079345703125 time for calcul the mask position with numpy : 0.03998875617980957 nb_pixel_total : 50485 time to create 1 rle with old method : 0.05547761917114258 time for calcul the mask position with numpy : 0.03518414497375488 nb_pixel_total : 95228 time to create 1 rle with old method : 0.10780930519104004 time for calcul the mask position with numpy : 0.0361018180847168 nb_pixel_total : 53613 time to create 1 rle with old method : 0.05973243713378906 time for calcul the mask position with numpy : 0.035895586013793945 nb_pixel_total : 7077 time to create 1 rle with old method : 0.00783395767211914 time for calcul the mask position with numpy : 0.04040050506591797 nb_pixel_total : 3051 time to create 1 rle with old method : 0.004184246063232422 time for calcul the mask position with numpy : 0.036409616470336914 nb_pixel_total : 53634 time to create 1 rle with old method : 0.061544179916381836 create new chi : 6.764314889907837 time to delete rle : 0.0012774467468261719 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 11773 TO DO : save crop sub photo not yet done ! save time : 8.176016807556152 nb_obj : 30 nb_hashtags : 7 time to prepare the origin masks : 12.720832347869873 time for calcul the mask position with numpy : 0.883812665939331 nb_pixel_total : 5702154 time to create 1 rle with new method : 0.3474140167236328 time for calcul the mask position with numpy : 0.029191017150878906 nb_pixel_total : 8702 time to create 1 rle with old method : 0.009606599807739258 time for calcul the mask position with numpy : 0.030357837677001953 nb_pixel_total : 108999 time to create 1 rle with old method : 0.12327384948730469 time for calcul the mask position with numpy : 0.028653383255004883 nb_pixel_total : 2307 time to create 1 rle with old method : 0.002662181854248047 time for calcul the mask position with numpy : 0.03306984901428223 nb_pixel_total : 226176 time to create 1 rle with new method : 1.0127167701721191 time for calcul the mask position with numpy : 0.029794692993164062 nb_pixel_total : 3684 time to create 1 rle with old method : 0.004365682601928711 time for calcul the mask position with numpy : 0.031577348709106445 nb_pixel_total : 182437 time to create 1 rle with new method : 0.3338735103607178 time for calcul the mask position with numpy : 0.02958226203918457 nb_pixel_total : 84986 time to create 1 rle with old method : 0.08885979652404785 time for calcul the mask position with numpy : 0.027809619903564453 nb_pixel_total : 14268 time to create 1 rle with old method : 0.015160322189331055 time for calcul the mask position with numpy : 0.02805948257446289 nb_pixel_total : 31319 time to create 1 rle with old method : 0.03531694412231445 time for calcul the mask position with numpy : 0.028423786163330078 nb_pixel_total : 2989 time to create 1 rle with old method : 0.0036220550537109375 time for calcul the mask position with numpy : 0.028167009353637695 nb_pixel_total : 23604 time to create 1 rle with old method : 0.02502894401550293 time for calcul the mask position with numpy : 0.02742481231689453 nb_pixel_total : 9183 time to create 1 rle with old method : 0.010064363479614258 time for calcul the mask position with numpy : 0.028034687042236328 nb_pixel_total : 164 time to create 1 rle with old method : 0.0002791881561279297 time for calcul the mask position with numpy : 0.027772188186645508 nb_pixel_total : 5563 time to create 1 rle with old method : 0.006128549575805664 time for calcul the mask position with numpy : 0.02865004539489746 nb_pixel_total : 97701 time to create 1 rle with old method : 0.10213494300842285 time for calcul the mask position with numpy : 0.027751922607421875 nb_pixel_total : 13106 time to create 1 rle with old method : 0.013559579849243164 time for calcul the mask position with numpy : 0.029030323028564453 nb_pixel_total : 66359 time to create 1 rle with old method : 0.07195401191711426 time for calcul the mask position with numpy : 0.030382394790649414 nb_pixel_total : 148275 time to create 1 rle with old method : 0.1529085636138916 time for calcul the mask position with numpy : 0.029933929443359375 nb_pixel_total : 36495 time to create 1 rle with old method : 0.041434526443481445 time for calcul the mask position with numpy : 0.028845787048339844 nb_pixel_total : 68697 time to create 1 rle with old method : 0.07189750671386719 time for calcul the mask position with numpy : 0.02918553352355957 nb_pixel_total : 84664 time to create 1 rle with old method : 0.08895754814147949 time for calcul the mask position with numpy : 0.028748512268066406 nb_pixel_total : 7427 time to create 1 rle with old method : 0.007862091064453125 time for calcul the mask position with numpy : 0.028584957122802734 nb_pixel_total : 10467 time to create 1 rle with old method : 0.011266469955444336 time for calcul the mask position with numpy : 0.030975818634033203 nb_pixel_total : 25086 time to create 1 rle with old method : 0.02666020393371582 time for calcul the mask position with numpy : 0.030025482177734375 nb_pixel_total : 3860 time to create 1 rle with old method : 0.00470733642578125 time for calcul the mask position with numpy : 0.03409171104431152 nb_pixel_total : 46760 time to create 1 rle with old method : 0.07211709022521973 time for calcul the mask position with numpy : 0.03288578987121582 nb_pixel_total : 13484 time to create 1 rle with old method : 0.01576828956604004 time for calcul the mask position with numpy : 0.032824039459228516 nb_pixel_total : 9584 time to create 1 rle with old method : 0.011090755462646484 time for calcul the mask position with numpy : 0.03560924530029297 nb_pixel_total : 10838 time to create 1 rle with old method : 0.015261650085449219 time for calcul the mask position with numpy : 0.033315420150756836 nb_pixel_total : 902 time to create 1 rle with old method : 0.0011551380157470703 create new chi : 4.6034440994262695 time to delete rle : 0.00202178955078125 batch 1 Loaded 38 chid ids of type : 3726 Number RLEs to save : 18426 TO DO : save crop sub photo not yet done ! save time : 26.369876384735107 nb_obj : 25 nb_hashtags : 8 time to prepare the origin masks : 24.807300567626953 time for calcul the mask position with numpy : 1.047600269317627 nb_pixel_total : 5888805 time to create 1 rle with new method : 0.930347204208374 time for calcul the mask position with numpy : 0.03393912315368652 nb_pixel_total : 242 time to create 1 rle with old method : 0.0004897117614746094 time for calcul the mask position with numpy : 0.03655552864074707 nb_pixel_total : 131539 time to create 1 rle with old method : 0.14258861541748047 time for calcul the mask position with numpy : 0.038031578063964844 nb_pixel_total : 61849 time to create 1 rle with old method : 0.08300423622131348 time for calcul the mask position with numpy : 0.03413963317871094 nb_pixel_total : 3983 time to create 1 rle with old method : 0.004206418991088867 time for calcul the mask position with numpy : 0.04092121124267578 nb_pixel_total : 73509 time to create 1 rle with old method : 0.07957887649536133 time for calcul the mask position with numpy : 0.03395390510559082 nb_pixel_total : 813 time to create 1 rle with old method : 0.0009322166442871094 time for calcul the mask position with numpy : 0.03591609001159668 nb_pixel_total : 1483 time to create 1 rle with old method : 0.0016732215881347656 time for calcul the mask position with numpy : 0.035475730895996094 nb_pixel_total : 49297 time to create 1 rle with old method : 0.054993391036987305 time for calcul the mask position with numpy : 0.04151654243469238 nb_pixel_total : 45865 time to create 1 rle with old method : 0.0528719425201416 time for calcul the mask position with numpy : 0.03771233558654785 nb_pixel_total : 107125 time to create 1 rle with old method : 0.14623045921325684 time for calcul the mask position with numpy : 0.03689885139465332 nb_pixel_total : 36825 time to create 1 rle with old method : 0.04076719284057617 time for calcul the mask position with numpy : 0.040805816650390625 nb_pixel_total : 75964 time to create 1 rle with old method : 0.0867619514465332 time for calcul the mask position with numpy : 0.03418993949890137 nb_pixel_total : 29100 time to create 1 rle with old method : 0.031754255294799805 time for calcul the mask position with numpy : 0.02260422706604004 nb_pixel_total : 45757 time to create 1 rle with old method : 0.051859378814697266 time for calcul the mask position with numpy : 0.035462141036987305 nb_pixel_total : 4568 time to create 1 rle with old method : 0.005602121353149414 time for calcul the mask position with numpy : 0.04530453681945801 nb_pixel_total : 197188 time to create 1 rle with new method : 0.6921393871307373 time for calcul the mask position with numpy : 0.03795671463012695 nb_pixel_total : 11361 time to create 1 rle with old method : 0.013042688369750977 time for calcul the mask position with numpy : 0.04222750663757324 nb_pixel_total : 22234 time to create 1 rle with old method : 0.030270814895629883 time for calcul the mask position with numpy : 0.03946733474731445 nb_pixel_total : 37649 time to create 1 rle with old method : 0.04763293266296387 time for calcul the mask position with numpy : 0.039061784744262695 nb_pixel_total : 60953 time to create 1 rle with old method : 0.06995511054992676 time for calcul the mask position with numpy : 0.03973722457885742 nb_pixel_total : 17812 time to create 1 rle with old method : 0.01959967613220215 time for calcul the mask position with numpy : 0.035012006759643555 nb_pixel_total : 75800 time to create 1 rle with old method : 0.08736515045166016 time for calcul the mask position with numpy : 0.03781723976135254 nb_pixel_total : 3544 time to create 1 rle with old method : 0.003946781158447266 time for calcul the mask position with numpy : 0.037030696868896484 nb_pixel_total : 41158 time to create 1 rle with old method : 0.04661679267883301 time for calcul the mask position with numpy : 0.022094249725341797 nb_pixel_total : 25817 time to create 1 rle with old method : 0.027947187423706055 create new chi : 4.774327039718628 time to delete rle : 0.005324602127075195 batch 1 Loaded 26 chid ids of type : 3726 Number RLEs to save : 16658 TO DO : save crop sub photo not yet done ! save time : 6.445210218429565 nb_obj : 28 nb_hashtags : 7 time to prepare the origin masks : 11.095908641815186 time for calcul the mask position with numpy : 0.9377553462982178 nb_pixel_total : 5423555 time to create 1 rle with new method : 1.9038832187652588 time for calcul the mask position with numpy : 0.03600049018859863 nb_pixel_total : 50596 time to create 1 rle with old method : 0.06846380233764648 time for calcul the mask position with numpy : 0.03355216979980469 nb_pixel_total : 90958 time to create 1 rle with old method : 0.10059356689453125 time for calcul the mask position with numpy : 0.028707027435302734 nb_pixel_total : 27777 time to create 1 rle with old method : 0.030962228775024414 time for calcul the mask position with numpy : 0.030698776245117188 nb_pixel_total : 2591 time to create 1 rle with old method : 0.0033638477325439453 time for calcul the mask position with numpy : 0.04310894012451172 nb_pixel_total : 462238 time to create 1 rle with new method : 0.905327558517456 time for calcul the mask position with numpy : 0.03486156463623047 nb_pixel_total : 102803 time to create 1 rle with old method : 0.1555469036102295 time for calcul the mask position with numpy : 0.03237795829772949 nb_pixel_total : 801 time to create 1 rle with old method : 0.0011293888092041016 time for calcul the mask position with numpy : 0.030079364776611328 nb_pixel_total : 22597 time to create 1 rle with old method : 0.024831056594848633 time for calcul the mask position with numpy : 0.03008413314819336 nb_pixel_total : 37159 time to create 1 rle with old method : 0.03994250297546387 time for calcul the mask position with numpy : 0.028861045837402344 nb_pixel_total : 9071 time to create 1 rle with old method : 0.010341644287109375 time for calcul the mask position with numpy : 0.028668642044067383 nb_pixel_total : 17829 time to create 1 rle with old method : 0.018610477447509766 time for calcul the mask position with numpy : 0.028182268142700195 nb_pixel_total : 1382 time to create 1 rle with old method : 0.0016665458679199219 time for calcul the mask position with numpy : 0.029839754104614258 nb_pixel_total : 108133 time to create 1 rle with old method : 0.11055850982666016 time for calcul the mask position with numpy : 0.028942584991455078 nb_pixel_total : 128268 time to create 1 rle with old method : 0.13192105293273926 time for calcul the mask position with numpy : 0.029656648635864258 nb_pixel_total : 52375 time to create 1 rle with old method : 0.05575823783874512 time for calcul the mask position with numpy : 0.028241395950317383 nb_pixel_total : 6171 time to create 1 rle with old method : 0.006226062774658203 time for calcul the mask position with numpy : 0.027451276779174805 nb_pixel_total : 48567 time to create 1 rle with old method : 0.05162692070007324 time for calcul the mask position with numpy : 0.028786420822143555 nb_pixel_total : 24920 time to create 1 rle with old method : 0.027849912643432617 time for calcul the mask position with numpy : 0.028308629989624023 nb_pixel_total : 1732 time to create 1 rle with old method : 0.0018949508666992188 time for calcul the mask position with numpy : 0.02988290786743164 nb_pixel_total : 51297 time to create 1 rle with old method : 0.07257509231567383 time for calcul the mask position with numpy : 0.029229164123535156 nb_pixel_total : 3028 time to create 1 rle with old method : 0.005096912384033203 time for calcul the mask position with numpy : 0.03241753578186035 nb_pixel_total : 8358 time to create 1 rle with old method : 0.009086132049560547 time for calcul the mask position with numpy : 0.03274655342102051 nb_pixel_total : 161893 time to create 1 rle with new method : 1.621561050415039 time for calcul the mask position with numpy : 0.04380035400390625 nb_pixel_total : 20908 time to create 1 rle with old method : 0.02870941162109375 time for calcul the mask position with numpy : 0.030396699905395508 nb_pixel_total : 99008 time to create 1 rle with old method : 0.1515340805053711 time for calcul the mask position with numpy : 0.032833099365234375 nb_pixel_total : 71553 time to create 1 rle with old method : 0.09340596199035645 time for calcul the mask position with numpy : 0.03501772880554199 nb_pixel_total : 8370 time to create 1 rle with old method : 0.011024236679077148 time for calcul the mask position with numpy : 0.03289484977722168 nb_pixel_total : 6302 time to create 1 rle with old method : 0.007443666458129883 create new chi : 7.595274209976196 time to delete rle : 0.003075122833251953 batch 1 Loaded 32 chid ids of type : 3726 Number RLEs to save : 19070 TO DO : save crop sub photo not yet done ! save time : 13.96894359588623 map_output_result : {1343023136: (0.0, 'Should be the crop_list due to order', 0), 1343023075: (0.0, 'Should be the crop_list due to order', 0), 1343022775: (0.0, 'Should be the crop_list due to order', 0), 1343022749: (0.0, 'Should be the crop_list due to order', 0), 1343022747: (0.0, 'Should be the crop_list due to order', 0), 1343022739: (0.0, 'Should be the crop_list due to order', 0), 1343022531: (0.0, 'Should be the crop_list due to order', 0), 1343022217: (0.0, 'Should be the crop_list due to order', 0), 1343022175: (0.0, 'Should be the crop_list due to order', 0), 1343022101: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 10 /1343023136.Didn't retrieve data . /1343023075.Didn't retrieve data . /1343022775.Didn't retrieve data . /1343022749.Didn't retrieve data . /1343022747.Didn't retrieve data . /1343022739.Didn't retrieve data . /1343022531.Didn't retrieve data . /1343022217.Didn't retrieve data . /1343022175.Didn't retrieve data . /1343022101.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023136', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023075', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022775', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022749', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022747', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022739', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022531', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022217', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022175', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022101', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.03149747848510742 save_final save missing photos in datou_result : time spend for datou_step_exec : 301.41973185539246 time spend to save output : 0.032067298889160156 total time spend for step 5 : 301.4517991542816 step6:crop_condition Tue Mar 11 15:28:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 228 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! About to upload 20 photos upload in portfolio : 4869462 init cache_photo without model_param we have 20 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703311_1396219 we have uploaded 20 photos in the portfolio 4869462 time of upload the photos Elapsed time : 14.509567499160767 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! About to upload 12 photos upload in portfolio : 4869462 init cache_photo without model_param we have 12 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703331_1396219 we have uploaded 12 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.540356397628784 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703337_1396219 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.029710531234741 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703341_1396219 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9625108242034912 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703345_1396219 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.422461986541748 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703352_1396219 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 7.178014278411865 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741703362_1396219 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.080152988433838 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 56 /1343071681Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071697Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071698Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071701Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071705Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071706Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071707Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071709Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071712Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071733Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071734Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071735Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071737Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071738Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071739Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071741Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071744Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071746Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071755Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071756Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071760Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071800Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071801Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071803Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071869Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071876Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071880Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071881Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071901Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071902Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343071903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023136', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023075', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022775', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022749', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022747', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022739', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022531', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022217', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022175', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022101', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 178 time used for this insertion : 1.4008569717407227 save_final save missing photos in datou_result : time spend for datou_step_exec : 79.72480034828186 time spend to save output : 1.407458782196045 total time spend for step 6 : 81.1322591304779 step7:ventilate_hashtags_in_portfolio Tue Mar 11 15:29:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 21291428 get user id for portfolio 21291428 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291428 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','papier','autre_refus','flou','Carton_brun','Teint_Dans_La_Masse','kraft','plastique','environnement','Carton_gris','mal_croppe','metal')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291428 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','papier','autre_refus','flou','Carton_brun','Teint_Dans_La_Masse','kraft','plastique','environnement','Carton_gris','mal_croppe','metal')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291428 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('cartonnette','papier','autre_refus','flou','Carton_brun','Teint_Dans_La_Masse','kraft','plastique','environnement','Carton_gris','mal_croppe','metal')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/21292097,21292098,21292099,21292100,21292101,21292103,21292104,21292105,21292106,21292108,21292109,21292110?tags=cartonnette,papier,autre_refus,flou,Carton_brun,Teint_Dans_La_Masse,kraft,plastique,environnement,Carton_gris,mal_croppe,metal Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 1 /21291428. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023136', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023075', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022775', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022749', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022747', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022739', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022531', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022217', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022175', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022101', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.018398284912109375 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.850809335708618 time spend to save output : 0.018705368041992188 total time spend for step 7 : 2.8695147037506104 step8:final Tue Mar 11 15:29:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1343023136: ('0.0812687785847433',), 1343023075: ('0.0812687785847433',), 1343022775: ('0.0812687785847433',), 1343022749: ('0.0812687785847433',), 1343022747: ('0.0812687785847433',), 1343022739: ('0.0812687785847433',), 1343022531: ('0.0812687785847433',), 1343022217: ('0.0812687785847433',), 1343022175: ('0.0812687785847433',), 1343022101: ('0.0812687785847433',)} new output for save of step final : {1343023136: ('0.0812687785847433',), 1343023075: ('0.0812687785847433',), 1343022775: ('0.0812687785847433',), 1343022749: ('0.0812687785847433',), 1343022747: ('0.0812687785847433',), 1343022739: ('0.0812687785847433',), 1343022531: ('0.0812687785847433',), 1343022217: ('0.0812687785847433',), 1343022175: ('0.0812687785847433',), 1343022101: ('0.0812687785847433',)} [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 10 /1343023136.Didn't retrieve data . /1343023075.Didn't retrieve data . /1343022775.Didn't retrieve data . /1343022749.Didn't retrieve data . /1343022747.Didn't retrieve data . /1343022739.Didn't retrieve data . /1343022531.Didn't retrieve data . /1343022217.Didn't retrieve data . /1343022175.Didn't retrieve data . /1343022101.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023136', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023075', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022775', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022749', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022747', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022739', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022531', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022217', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022175', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022101', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.14023661613464355 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10187506675720215 time spend to save output : 0.1408863067626953 total time spend for step 8 : 0.24276137351989746 step9:velours_tree Tue Mar 11 15:29:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.038030147552490234 time spend to save output : 3.790855407714844e-05 total time spend for step 9 : 0.03806805610656738 step10:send_mail_cod Tue Mar 11 15:29:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P21291428_11-03-2025_15_29_45.pdf 21292097 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212920971741703385 21292099 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212920991741703386 21292100 imagette212921001741703387 21292104 change filename to text .change filename to text .change filename to text .change filename to text .imagette212921041741703387 21292105 change filename to text .change filename to text .change filename to text .change filename to text .imagette212921051741703392 21292109 imagette212921091741703392 21292110 imagette212921101741703392 21292101 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212921011741703392 21292103 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212921031741703393 21292108 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette212921081741703395 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21291428 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21292097,21292098,21292099,21292100,21292101,21292103,21292104,21292105,21292106,21292108,21292109,21292110?tags=cartonnette,papier,autre_refus,flou,Carton_brun,Teint_Dans_La_Masse,kraft,plastique,environnement,Carton_gris,mal_croppe,metal your option no_mail is active, we will not send the real mail to your client args[1343023136] : ('0.0812687785847433',) no score found for photo 1343023136 We are sending mail with results at report@fotonower.com args[1343023075] : ('0.0812687785847433',) no score found for photo 1343023075 We are sending mail with results at report@fotonower.com args[1343022775] : ('0.0812687785847433',) no score found for photo 1343022775 We are sending mail with results at report@fotonower.com args[1343022749] : ('0.0812687785847433',) no score found for photo 1343022749 We are sending mail with results at report@fotonower.com args[1343022747] : ('0.0812687785847433',) no score found for photo 1343022747 We are sending mail with results at report@fotonower.com args[1343022739] : ('0.0812687785847433',) no score found for photo 1343022739 We are sending mail with results at report@fotonower.com args[1343022531] : ('0.0812687785847433',) no score found for photo 1343022531 We are sending mail with results at report@fotonower.com args[1343022217] : ('0.0812687785847433',) no score found for photo 1343022217 We are sending mail with results at report@fotonower.com args[1343022175] : ('0.0812687785847433',) no score found for photo 1343022175 We are sending mail with results at report@fotonower.com args[1343022101] : ('0.0812687785847433',) no score found for photo 1343022101 We are sending mail with results at report@fotonower.com refus_total : 0.0812687785847433 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21291428 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1343021425,1343021430,1343021654,1343021678,1343022749,1343022747,1343021377,1343022217,1343022531,1343022739,1343023075,1343022775,1343022175,1343022101,1343023136,1343022083,1343022053,1343021603,1343021431,1343021403) Found this number of photos: 20 begin to download photo : 1343021425 begin to download photo : 1343022747 begin to download photo : 1343023075 begin to download photo : 1343022083 download finish for photo 1343022083 begin to download photo : 1343022053 download finish for photo 1343023075 begin to download photo : 1343022775 download finish for photo 1343021425 begin to download photo : 1343021430 download finish for photo 1343022747 begin to download photo : 1343021377 download finish for photo 1343022053 begin to download photo : 1343021603 download finish for photo 1343021430 begin to download photo : 1343021654 download finish for photo 1343022775 begin to download photo : 1343022175 download finish for photo 1343021377 begin to download photo : 1343022217 download finish for photo 1343021603 begin to download photo : 1343021431 download finish for photo 1343021654 begin to download photo : 1343021678 download finish for photo 1343022175 begin to download photo : 1343022101 download finish for photo 1343022217 begin to download photo : 1343022531 download finish for photo 1343021431 begin to download photo : 1343021403 download finish for photo 1343021678 begin to download photo : 1343022749 download finish for photo 1343022101 begin to download photo : 1343023136 download finish for photo 1343021403 download finish for photo 1343023136 download finish for photo 1343022531 begin to download photo : 1343022739 download finish for photo 1343022749 download finish for photo 1343022739 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_29_45.pdf results_Auto_P21291428_11-03-2025_15_29_45.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_29_45.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','21291428','results_Auto_P21291428_11-03-2025_15_29_45.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_29_45.pdf','pdf','','0.99','0.0812687785847433') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023136', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023075', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022775', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022749', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022747', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022739', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022531', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022217', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022175', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022101', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013529300689697266 save_final save missing photos in datou_result : time spend for datou_step_exec : 15.187511682510376 time spend to save output : 0.013785839080810547 total time spend for step 10 : 15.201297521591187 step11:split_time_score Tue Mar 11 15:30:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('13', 20),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11032025 21291428 Nombre de photos uploadées : 20 / 23040 (0%) 11032025 21291428 Nombre de photos taguées (types de déchets): 0 / 20 (0%) 11032025 21291428 Nombre de photos taguées (volume) : 0 / 20 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 6.4373016357421875e-06 ???????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0042688846588134766 elapsed_time : insert_dashboard_record_day_entry 0.709338903427124 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291426 order by id desc limit 1 Qualite : 0.07848604537653218 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268995_11-03-2025_10_04_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268995 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268995 AND mptpi.`type`=4207 To do Qualite : 0.21508235889842053 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268996_11-03-2025_09_21_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268996 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268996 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268997_11-03-2025_09_36_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268997 order by id desc limit 1 Qualite : 0.2996323014615577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268998_11-03-2025_09_21_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268998 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268998 AND mptpi.`type`=4200 To do Qualite : 0.059107923268759154 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21269000_11-03-2025_09_44_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21269000 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21269000 AND mptpi.`type`=4209 To do Qualite : 0.09529821171173529 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21269001_11-03-2025_09_17_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21269001 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21269001 AND mptpi.`type`=3726 To do Qualite : 0.12370809963529884 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21270945_11-03-2025_12_54_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21270945 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21270945 AND mptpi.`type`=3327 To do Qualite : 0.044100679011939585 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21276914_11-03-2025_11_38_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21276914 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21276914 AND mptpi.`type`=4207 To do Qualite : 0.04798165055335807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21276915_11-03-2025_12_41_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21276915 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21276915 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290022 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290047 order by id desc limit 1 Qualite : 0.24539234952758107 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282296_11-03-2025_13_26_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282296 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282296 AND mptpi.`type`=3594 To do Qualite : 0.23651912322606505 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282297_11-03-2025_14_28_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282297 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282297 AND mptpi.`type`=3594 To do Qualite : 0.30571381203013953 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282298_11-03-2025_13_24_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282298 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282298 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282299_11-03-2025_12_43_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282299 order by id desc limit 1 Qualite : 0.3846979317267506 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282300_11-03-2025_13_27_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282300 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282300 AND mptpi.`type`=4200 To do Qualite : 0.3844764824714978 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282301_11-03-2025_13_46_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282301 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282301 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282303_11-03-2025_12_44_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282303 order by id desc limit 1 Qualite : 0.21547889555127603 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290048_11-03-2025_15_09_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290048 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290048 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290049 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290050 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290051 order by id desc limit 1 Qualite : 0.04833780284360806 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290052_11-03-2025_15_05_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290052 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290052 AND mptpi.`type`=4207 To do Qualite : 0.0812687785847433 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_29_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291428 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291428 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291429 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291431 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11032025': {'nb_upload': 20, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1343023136, 1343023075, 1343022775, 1343022749, 1343022747, 1343022739, 1343022531, 1343022217, 1343022175, 1343022101] Looping around the photos to save general results len do output : 1 /21291428Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023136', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343023075', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022775', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022749', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022747', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022739', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022531', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022217', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022175', None, None, None, None, None, '2658518') ('3459', None, None, None, None, None, None, None, '2658518') ('3459', '21291428', '1343022101', None, None, None, None, None, '2658518') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.013264894485473633 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.236814260482788 time spend to save output : 0.013498783111572266 total time spend for step 11 : 10.25031304359436 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 212.99user 177.41system 9:46.31elapsed 66%CPU (0avgtext+0avgdata 4747076maxresident)k 354000inputs+152752outputs (10383major+15901965minor)pagefaults 0swaps