python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2576476 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 22>10 please execute split_portfolio.py -i 21307714 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2659516'] with mtr_portfolio_ids : ['21307714'] and first list_photo_ids : [] new path : /proc/2576476/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 2.421247959136963 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Mar 11 20:20:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 9707 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-03-11 20:20:34.881906: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-03-11 20:20:34.911138: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-03-11 20:20:34.913411: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe88c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-03-11 20:20:34.913471: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-03-11 20:20:34.917782: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-03-11 20:20:35.160837: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x121e8210 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-03-11 20:20:35.160918: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-03-11 20:20:35.162302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 20:20:35.162715: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 20:20:35.165450: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 20:20:35.168343: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 20:20:35.168821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 20:20:35.172074: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 20:20:35.173530: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 20:20:35.179924: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 20:20:35.181521: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 20:20:35.181631: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 20:20:35.182445: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 20:20:35.182465: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 20:20:35.182475: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 20:20:35.183917: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8982 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-03-11 20:20:35.547201: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 20:20:35.547337: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 20:20:35.547366: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 20:20:35.547388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 20:20:35.547403: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 20:20:35.547418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 20:20:35.547433: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 20:20:35.547448: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 20:20:35.548823: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 20:20:35.550254: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-03-11 20:20:35.550287: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-03-11 20:20:35.550302: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 20:20:35.550316: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-03-11 20:20:35.550330: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-03-11 20:20:35.550345: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-03-11 20:20:35.550359: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-03-11 20:20:35.550377: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-03-11 20:20:35.551833: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-03-11 20:20:35.551878: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-03-11 20:20:35.551888: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-03-11 20:20:35.551895: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-03-11 20:20:35.553344: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8982 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-03-11 20:20:46.291208: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-03-11 20:20:46.501077: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.64010 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 3 Detection mask done ! Trying to reset tf kernel 2577285 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 4639 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 9928 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.15358734130859375 nb_pixel_total : 3226801 time to create 1 rle with new method : 1.6588339805603027 length of segment : 3083 time for calcul the mask position with numpy : 0.0003044605255126953 nb_pixel_total : 8334 time to create 1 rle with old method : 0.010204553604125977 length of segment : 159 time for calcul the mask position with numpy : 0.0008387565612792969 nb_pixel_total : 37961 time to create 1 rle with old method : 0.04199051856994629 length of segment : 288 time for calcul the mask position with numpy : 0.001990795135498047 nb_pixel_total : 124132 time to create 1 rle with old method : 0.14905333518981934 length of segment : 303 time for calcul the mask position with numpy : 0.0009088516235351562 nb_pixel_total : 20618 time to create 1 rle with old method : 0.029864788055419922 length of segment : 201 time for calcul the mask position with numpy : 0.1537473201751709 nb_pixel_total : 589476 time to create 1 rle with new method : 0.06876587867736816 length of segment : 1197 time for calcul the mask position with numpy : 0.0013818740844726562 nb_pixel_total : 37093 time to create 1 rle with old method : 0.048894643783569336 length of segment : 562 time for calcul the mask position with numpy : 0.0010962486267089844 nb_pixel_total : 37125 time to create 1 rle with old method : 0.0546417236328125 length of segment : 378 time for calcul the mask position with numpy : 0.001035928726196289 nb_pixel_total : 42276 time to create 1 rle with old method : 0.0483396053314209 length of segment : 340 time for calcul the mask position with numpy : 0.0002703666687011719 nb_pixel_total : 9856 time to create 1 rle with old method : 0.012001276016235352 length of segment : 192 time for calcul the mask position with numpy : 0.00030517578125 nb_pixel_total : 11004 time to create 1 rle with old method : 0.01334238052368164 length of segment : 185 time for calcul the mask position with numpy : 0.005402565002441406 nb_pixel_total : 202066 time to create 1 rle with new method : 0.014533042907714844 length of segment : 539 time for calcul the mask position with numpy : 0.00032782554626464844 nb_pixel_total : 3180 time to create 1 rle with old method : 0.0038716793060302734 length of segment : 125 time for calcul the mask position with numpy : 0.002054929733276367 nb_pixel_total : 41782 time to create 1 rle with old method : 0.05755877494812012 length of segment : 350 time for calcul the mask position with numpy : 0.0022270679473876953 nb_pixel_total : 88812 time to create 1 rle with old method : 0.10475516319274902 length of segment : 417 time for calcul the mask position with numpy : 0.0027043819427490234 nb_pixel_total : 121986 time to create 1 rle with old method : 0.16911578178405762 length of segment : 358 time for calcul the mask position with numpy : 0.00025773048400878906 nb_pixel_total : 1610 time to create 1 rle with old method : 0.00716853141784668 length of segment : 64 time for calcul the mask position with numpy : 0.0012426376342773438 nb_pixel_total : 22182 time to create 1 rle with old method : 0.0393528938293457 length of segment : 184 time for calcul the mask position with numpy : 0.010206937789916992 nb_pixel_total : 232732 time to create 1 rle with new method : 0.035279035568237305 length of segment : 800 time for calcul the mask position with numpy : 0.0009222030639648438 nb_pixel_total : 31293 time to create 1 rle with old method : 0.0371859073638916 length of segment : 241 time for calcul the mask position with numpy : 0.0004780292510986328 nb_pixel_total : 9879 time to create 1 rle with old method : 0.012138843536376953 length of segment : 164 time for calcul the mask position with numpy : 0.0021545886993408203 nb_pixel_total : 88346 time to create 1 rle with old method : 0.10609698295593262 length of segment : 267 time for calcul the mask position with numpy : 0.004891633987426758 nb_pixel_total : 218363 time to create 1 rle with new method : 0.009514808654785156 length of segment : 557 time for calcul the mask position with numpy : 0.0008130073547363281 nb_pixel_total : 16395 time to create 1 rle with old method : 0.020853519439697266 length of segment : 194 time for calcul the mask position with numpy : 0.001262664794921875 nb_pixel_total : 36117 time to create 1 rle with old method : 0.04224848747253418 length of segment : 225 time for calcul the mask position with numpy : 0.0058135986328125 nb_pixel_total : 133549 time to create 1 rle with old method : 0.1534740924835205 length of segment : 453 time for calcul the mask position with numpy : 0.001190185546875 nb_pixel_total : 48217 time to create 1 rle with old method : 0.055860042572021484 length of segment : 193 time for calcul the mask position with numpy : 0.0005967617034912109 nb_pixel_total : 17493 time to create 1 rle with old method : 0.020735740661621094 length of segment : 176 time for calcul the mask position with numpy : 0.00032973289489746094 nb_pixel_total : 7272 time to create 1 rle with old method : 0.009163618087768555 length of segment : 75 time for calcul the mask position with numpy : 0.003133058547973633 nb_pixel_total : 100194 time to create 1 rle with old method : 0.11595845222473145 length of segment : 582 time for calcul the mask position with numpy : 0.0002396106719970703 nb_pixel_total : 3590 time to create 1 rle with old method : 0.004450798034667969 length of segment : 62 time for calcul the mask position with numpy : 0.0005562305450439453 nb_pixel_total : 6329 time to create 1 rle with old method : 0.007550716400146484 length of segment : 170 time for calcul the mask position with numpy : 0.000911712646484375 nb_pixel_total : 10640 time to create 1 rle with old method : 0.012595891952514648 length of segment : 235 time for calcul the mask position with numpy : 0.0002448558807373047 nb_pixel_total : 3404 time to create 1 rle with old method : 0.004136800765991211 length of segment : 85 time for calcul the mask position with numpy : 0.00047016143798828125 nb_pixel_total : 12681 time to create 1 rle with old method : 0.016197681427001953 length of segment : 135 time for calcul the mask position with numpy : 0.0016865730285644531 nb_pixel_total : 37207 time to create 1 rle with old method : 0.04191946983337402 length of segment : 210 time for calcul the mask position with numpy : 0.00013709068298339844 nb_pixel_total : 696 time to create 1 rle with old method : 0.000995635986328125 length of segment : 20 time for calcul the mask position with numpy : 0.000186920166015625 nb_pixel_total : 1704 time to create 1 rle with old method : 0.0022203922271728516 length of segment : 59 time for calcul the mask position with numpy : 0.0019068717956542969 nb_pixel_total : 64412 time to create 1 rle with old method : 0.07390832901000977 length of segment : 493 time for calcul the mask position with numpy : 0.0014729499816894531 nb_pixel_total : 58713 time to create 1 rle with old method : 0.06519126892089844 length of segment : 486 time for calcul the mask position with numpy : 0.00016069412231445312 nb_pixel_total : 1372 time to create 1 rle with old method : 0.0016369819641113281 length of segment : 45 time for calcul the mask position with numpy : 0.005718708038330078 nb_pixel_total : 21870 time to create 1 rle with old method : 0.026969432830810547 length of segment : 299 time for calcul the mask position with numpy : 0.001163482666015625 nb_pixel_total : 23529 time to create 1 rle with old method : 0.026673078536987305 length of segment : 427 time for calcul the mask position with numpy : 0.0012471675872802734 nb_pixel_total : 37600 time to create 1 rle with old method : 0.04291033744812012 length of segment : 224 time for calcul the mask position with numpy : 0.001325368881225586 nb_pixel_total : 38034 time to create 1 rle with old method : 0.04380464553833008 length of segment : 338 time for calcul the mask position with numpy : 0.0023653507232666016 nb_pixel_total : 71959 time to create 1 rle with old method : 0.07908487319946289 length of segment : 397 time for calcul the mask position with numpy : 0.0005156993865966797 nb_pixel_total : 11335 time to create 1 rle with old method : 0.013377189636230469 length of segment : 178 time for calcul the mask position with numpy : 0.0007901191711425781 nb_pixel_total : 23590 time to create 1 rle with old method : 0.02646803855895996 length of segment : 226 time for calcul the mask position with numpy : 0.0013976097106933594 nb_pixel_total : 35787 time to create 1 rle with old method : 0.04117012023925781 length of segment : 278 time for calcul the mask position with numpy : 0.0035669803619384766 nb_pixel_total : 163444 time to create 1 rle with new method : 0.005277872085571289 length of segment : 522 time for calcul the mask position with numpy : 0.0005810260772705078 nb_pixel_total : 13074 time to create 1 rle with old method : 0.0149078369140625 length of segment : 133 time for calcul the mask position with numpy : 0.00020313262939453125 nb_pixel_total : 2537 time to create 1 rle with old method : 0.0030639171600341797 length of segment : 95 time for calcul the mask position with numpy : 0.0006992816925048828 nb_pixel_total : 19657 time to create 1 rle with old method : 0.023113489151000977 length of segment : 185 time for calcul the mask position with numpy : 0.0013155937194824219 nb_pixel_total : 17646 time to create 1 rle with old method : 0.02156853675842285 length of segment : 409 time for calcul the mask position with numpy : 0.0006425380706787109 nb_pixel_total : 17876 time to create 1 rle with old method : 0.020848989486694336 length of segment : 194 time for calcul the mask position with numpy : 0.00026679039001464844 nb_pixel_total : 1647 time to create 1 rle with old method : 0.002150297164916992 length of segment : 105 time for calcul the mask position with numpy : 0.0006086826324462891 nb_pixel_total : 26202 time to create 1 rle with old method : 0.030843496322631836 length of segment : 297 time for calcul the mask position with numpy : 0.00045800209045410156 nb_pixel_total : 10468 time to create 1 rle with old method : 0.01750779151916504 length of segment : 130 time for calcul the mask position with numpy : 0.0018239021301269531 nb_pixel_total : 24650 time to create 1 rle with old method : 0.030295372009277344 length of segment : 409 time for calcul the mask position with numpy : 0.00043201446533203125 nb_pixel_total : 10270 time to create 1 rle with old method : 0.012745380401611328 length of segment : 155 time for calcul the mask position with numpy : 0.0007741451263427734 nb_pixel_total : 16815 time to create 1 rle with old method : 0.022931337356567383 length of segment : 209 time for calcul the mask position with numpy : 0.005703449249267578 nb_pixel_total : 103197 time to create 1 rle with old method : 0.13980412483215332 length of segment : 684 time for calcul the mask position with numpy : 0.0018763542175292969 nb_pixel_total : 57113 time to create 1 rle with old method : 0.07071638107299805 length of segment : 422 time for calcul the mask position with numpy : 0.0002307891845703125 nb_pixel_total : 2235 time to create 1 rle with old method : 0.0027947425842285156 length of segment : 72 time for calcul the mask position with numpy : 0.001961231231689453 nb_pixel_total : 82993 time to create 1 rle with old method : 0.09591555595397949 length of segment : 465 time for calcul the mask position with numpy : 0.002258777618408203 nb_pixel_total : 43472 time to create 1 rle with old method : 0.06889009475708008 length of segment : 310 time for calcul the mask position with numpy : 0.0002865791320800781 nb_pixel_total : 4663 time to create 1 rle with old method : 0.005596160888671875 length of segment : 54 time for calcul the mask position with numpy : 0.004414796829223633 nb_pixel_total : 145685 time to create 1 rle with old method : 0.16350507736206055 length of segment : 406 time for calcul the mask position with numpy : 0.0033867359161376953 nb_pixel_total : 132981 time to create 1 rle with old method : 0.1912071704864502 length of segment : 336 time for calcul the mask position with numpy : 0.0014526844024658203 nb_pixel_total : 51244 time to create 1 rle with old method : 0.06299901008605957 length of segment : 366 time for calcul the mask position with numpy : 0.001615285873413086 nb_pixel_total : 36673 time to create 1 rle with old method : 0.0664072036743164 length of segment : 230 time for calcul the mask position with numpy : 0.0026292800903320312 nb_pixel_total : 136845 time to create 1 rle with old method : 0.19890975952148438 length of segment : 372 time for calcul the mask position with numpy : 0.0004825592041015625 nb_pixel_total : 6365 time to create 1 rle with old method : 0.008948087692260742 length of segment : 115 time for calcul the mask position with numpy : 0.005054950714111328 nb_pixel_total : 134168 time to create 1 rle with old method : 0.15419554710388184 length of segment : 720 time for calcul the mask position with numpy : 0.0005156993865966797 nb_pixel_total : 9617 time to create 1 rle with old method : 0.011934280395507812 length of segment : 112 time for calcul the mask position with numpy : 0.002081632614135742 nb_pixel_total : 70945 time to create 1 rle with old method : 0.08212876319885254 length of segment : 325 time for calcul the mask position with numpy : 0.0017535686492919922 nb_pixel_total : 25287 time to create 1 rle with old method : 0.03046894073486328 length of segment : 193 time for calcul the mask position with numpy : 0.0070514678955078125 nb_pixel_total : 197248 time to create 1 rle with new method : 0.014404773712158203 length of segment : 804 time for calcul the mask position with numpy : 0.0015125274658203125 nb_pixel_total : 39771 time to create 1 rle with old method : 0.04732680320739746 length of segment : 377 time for calcul the mask position with numpy : 0.0017392635345458984 nb_pixel_total : 20716 time to create 1 rle with old method : 0.02512335777282715 length of segment : 275 time for calcul the mask position with numpy : 0.00046253204345703125 nb_pixel_total : 8586 time to create 1 rle with old method : 0.010457754135131836 length of segment : 109 time for calcul the mask position with numpy : 0.0028228759765625 nb_pixel_total : 105901 time to create 1 rle with old method : 0.12032938003540039 length of segment : 729 time for calcul the mask position with numpy : 0.001165628433227539 nb_pixel_total : 25618 time to create 1 rle with old method : 0.029679298400878906 length of segment : 198 time for calcul the mask position with numpy : 0.0002009868621826172 nb_pixel_total : 3943 time to create 1 rle with old method : 0.00484156608581543 length of segment : 80 time for calcul the mask position with numpy : 0.0005545616149902344 nb_pixel_total : 15889 time to create 1 rle with old method : 0.018497705459594727 length of segment : 175 time for calcul the mask position with numpy : 0.0014545917510986328 nb_pixel_total : 49656 time to create 1 rle with old method : 0.058258771896362305 length of segment : 260 time for calcul the mask position with numpy : 0.0016906261444091797 nb_pixel_total : 64088 time to create 1 rle with old method : 0.07524895668029785 length of segment : 314 time for calcul the mask position with numpy : 0.002544879913330078 nb_pixel_total : 55525 time to create 1 rle with old method : 0.06322312355041504 length of segment : 417 time for calcul the mask position with numpy : 0.0014801025390625 nb_pixel_total : 59561 time to create 1 rle with old method : 0.07118082046508789 length of segment : 432 time for calcul the mask position with numpy : 0.0017604827880859375 nb_pixel_total : 52942 time to create 1 rle with old method : 0.06321430206298828 length of segment : 301 time for calcul the mask position with numpy : 0.002657651901245117 nb_pixel_total : 92698 time to create 1 rle with old method : 0.10823440551757812 length of segment : 312 time for calcul the mask position with numpy : 0.0026862621307373047 nb_pixel_total : 74961 time to create 1 rle with old method : 0.09292864799499512 length of segment : 244 time for calcul the mask position with numpy : 0.0025420188903808594 nb_pixel_total : 91261 time to create 1 rle with old method : 0.10290670394897461 length of segment : 433 time for calcul the mask position with numpy : 0.00023865699768066406 nb_pixel_total : 4960 time to create 1 rle with old method : 0.006179094314575195 length of segment : 58 time for calcul the mask position with numpy : 0.0001983642578125 nb_pixel_total : 5005 time to create 1 rle with old method : 0.006493806838989258 length of segment : 44 time for calcul the mask position with numpy : 0.00013709068298339844 nb_pixel_total : 2372 time to create 1 rle with old method : 0.0030384063720703125 length of segment : 52 time for calcul the mask position with numpy : 0.0011723041534423828 nb_pixel_total : 69795 time to create 1 rle with old method : 0.08022904396057129 length of segment : 222 time for calcul the mask position with numpy : 0.0002410411834716797 nb_pixel_total : 5374 time to create 1 rle with old method : 0.00673365592956543 length of segment : 61 time for calcul the mask position with numpy : 0.0006279945373535156 nb_pixel_total : 24007 time to create 1 rle with old method : 0.02874898910522461 length of segment : 216 time for calcul the mask position with numpy : 0.0003027915954589844 nb_pixel_total : 14512 time to create 1 rle with old method : 0.0174100399017334 length of segment : 156 time for calcul the mask position with numpy : 0.0005450248718261719 nb_pixel_total : 14716 time to create 1 rle with old method : 0.017896175384521484 length of segment : 145 time for calcul the mask position with numpy : 0.003209352493286133 nb_pixel_total : 101450 time to create 1 rle with old method : 0.12074041366577148 length of segment : 508 time for calcul the mask position with numpy : 0.003985881805419922 nb_pixel_total : 145557 time to create 1 rle with old method : 0.16367006301879883 length of segment : 469 time for calcul the mask position with numpy : 0.0005459785461425781 nb_pixel_total : 20174 time to create 1 rle with old method : 0.023391008377075195 length of segment : 144 time for calcul the mask position with numpy : 0.0007178783416748047 nb_pixel_total : 16224 time to create 1 rle with old method : 0.023946762084960938 length of segment : 187 time for calcul the mask position with numpy : 0.0017406940460205078 nb_pixel_total : 52435 time to create 1 rle with old method : 0.06083106994628906 length of segment : 236 time for calcul the mask position with numpy : 0.00028896331787109375 nb_pixel_total : 3984 time to create 1 rle with old method : 0.004980564117431641 length of segment : 87 time for calcul the mask position with numpy : 0.002290010452270508 nb_pixel_total : 79519 time to create 1 rle with old method : 0.1165463924407959 length of segment : 300 time for calcul the mask position with numpy : 0.0002887248992919922 nb_pixel_total : 2157 time to create 1 rle with old method : 0.0028100013732910156 length of segment : 65 time for calcul the mask position with numpy : 0.005493640899658203 nb_pixel_total : 255511 time to create 1 rle with new method : 0.009944915771484375 length of segment : 587 time for calcul the mask position with numpy : 0.0012824535369873047 nb_pixel_total : 29801 time to create 1 rle with old method : 0.04000282287597656 length of segment : 228 time for calcul the mask position with numpy : 0.0012836456298828125 nb_pixel_total : 78920 time to create 1 rle with old method : 0.09240841865539551 length of segment : 304 time for calcul the mask position with numpy : 0.003330230712890625 nb_pixel_total : 107439 time to create 1 rle with old method : 0.1329176425933838 length of segment : 354 time for calcul the mask position with numpy : 0.004604816436767578 nb_pixel_total : 155954 time to create 1 rle with new method : 0.008551836013793945 length of segment : 693 time for calcul the mask position with numpy : 0.0005168914794921875 nb_pixel_total : 14207 time to create 1 rle with old method : 0.020917415618896484 length of segment : 117 time for calcul the mask position with numpy : 0.0003933906555175781 nb_pixel_total : 8585 time to create 1 rle with old method : 0.014933109283447266 length of segment : 111 time for calcul the mask position with numpy : 0.006463289260864258 nb_pixel_total : 278799 time to create 1 rle with new method : 0.008725881576538086 length of segment : 623 time for calcul the mask position with numpy : 0.002748250961303711 nb_pixel_total : 87854 time to create 1 rle with old method : 0.10070943832397461 length of segment : 261 time for calcul the mask position with numpy : 0.0016245841979980469 nb_pixel_total : 85331 time to create 1 rle with old method : 0.09793901443481445 length of segment : 292 time for calcul the mask position with numpy : 0.0019459724426269531 nb_pixel_total : 74757 time to create 1 rle with old method : 0.0907742977142334 length of segment : 657 time for calcul the mask position with numpy : 0.00012493133544921875 nb_pixel_total : 2813 time to create 1 rle with old method : 0.003552675247192383 length of segment : 61 time for calcul the mask position with numpy : 0.00023245811462402344 nb_pixel_total : 10138 time to create 1 rle with old method : 0.012704133987426758 length of segment : 134 time for calcul the mask position with numpy : 0.003244638442993164 nb_pixel_total : 91867 time to create 1 rle with old method : 0.10534524917602539 length of segment : 476 time for calcul the mask position with numpy : 0.0017521381378173828 nb_pixel_total : 42959 time to create 1 rle with old method : 0.051648855209350586 length of segment : 411 time for calcul the mask position with numpy : 0.008877992630004883 nb_pixel_total : 278790 time to create 1 rle with new method : 0.013228416442871094 length of segment : 626 time for calcul the mask position with numpy : 0.0008096694946289062 nb_pixel_total : 25049 time to create 1 rle with old method : 0.029114246368408203 length of segment : 159 time for calcul the mask position with numpy : 0.0001766681671142578 nb_pixel_total : 2011 time to create 1 rle with old method : 0.002627849578857422 length of segment : 57 time for calcul the mask position with numpy : 0.004546165466308594 nb_pixel_total : 152598 time to create 1 rle with new method : 0.006082296371459961 length of segment : 461 time for calcul the mask position with numpy : 0.003180980682373047 nb_pixel_total : 95946 time to create 1 rle with old method : 0.10788249969482422 length of segment : 551 time for calcul the mask position with numpy : 0.0006327629089355469 nb_pixel_total : 14055 time to create 1 rle with old method : 0.015549421310424805 length of segment : 158 time for calcul the mask position with numpy : 0.00015091896057128906 nb_pixel_total : 1326 time to create 1 rle with old method : 0.0015246868133544922 length of segment : 55 time for calcul the mask position with numpy : 0.00015878677368164062 nb_pixel_total : 2400 time to create 1 rle with old method : 0.002765178680419922 length of segment : 127 time for calcul the mask position with numpy : 0.001622915267944336 nb_pixel_total : 42581 time to create 1 rle with old method : 0.055223703384399414 length of segment : 300 time for calcul the mask position with numpy : 0.0007061958312988281 nb_pixel_total : 12665 time to create 1 rle with old method : 0.016307353973388672 length of segment : 176 time for calcul the mask position with numpy : 0.0023353099822998047 nb_pixel_total : 64448 time to create 1 rle with old method : 0.07214093208312988 length of segment : 377 time for calcul the mask position with numpy : 0.002803802490234375 nb_pixel_total : 71970 time to create 1 rle with old method : 0.08336639404296875 length of segment : 448 time for calcul the mask position with numpy : 0.0006058216094970703 nb_pixel_total : 25413 time to create 1 rle with old method : 0.029285192489624023 length of segment : 146 time for calcul the mask position with numpy : 0.001969575881958008 nb_pixel_total : 109631 time to create 1 rle with old method : 0.13247036933898926 length of segment : 582 time for calcul the mask position with numpy : 0.002118349075317383 nb_pixel_total : 42196 time to create 1 rle with old method : 0.04987192153930664 length of segment : 270 time for calcul the mask position with numpy : 0.00083160400390625 nb_pixel_total : 17085 time to create 1 rle with old method : 0.021017789840698242 length of segment : 174 time for calcul the mask position with numpy : 0.00012564659118652344 nb_pixel_total : 986 time to create 1 rle with old method : 0.0013191699981689453 length of segment : 29 time for calcul the mask position with numpy : 0.0017592906951904297 nb_pixel_total : 64274 time to create 1 rle with old method : 0.07558083534240723 length of segment : 299 time for calcul the mask position with numpy : 0.006448507308959961 nb_pixel_total : 89015 time to create 1 rle with old method : 0.10226678848266602 length of segment : 561 time for calcul the mask position with numpy : 0.0015881061553955078 nb_pixel_total : 32429 time to create 1 rle with old method : 0.0385739803314209 length of segment : 295 time for calcul the mask position with numpy : 0.0029561519622802734 nb_pixel_total : 86095 time to create 1 rle with old method : 0.09896588325500488 length of segment : 601 time for calcul the mask position with numpy : 0.0003561973571777344 nb_pixel_total : 10472 time to create 1 rle with old method : 0.01280975341796875 length of segment : 91 time for calcul the mask position with numpy : 0.0004050731658935547 nb_pixel_total : 7033 time to create 1 rle with old method : 0.00871133804321289 length of segment : 97 time for calcul the mask position with numpy : 0.0008051395416259766 nb_pixel_total : 42642 time to create 1 rle with old method : 0.04995298385620117 length of segment : 189 time for calcul the mask position with numpy : 0.5685060024261475 nb_pixel_total : 5010588 time to create 1 rle with new method : 0.722813606262207 length of segment : 3416 time spent for convertir_results : 19.169662714004517 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 149 chid ids of type : 3663 ++Number RLEs to save : 49208 save missing photos in datou_result : time spend for datou_step_exec : 72.70248985290527 time spend to save output : 65.17812657356262 total time spend for step 1 : 137.8806164264679 step2:crop_condition Tue Mar 11 20:22:49 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 149 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 43 About to insert : list_path_to_insert length 43 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1343155425, 1343155343, 1343155302, 1343155245, 1343154814, 1343154774, 1343154734, 1343154049, 1343154014, 1343140519] Looping around the photos to save general results len do output : 84 /-3706305475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305545Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305550Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305546Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305562Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305577Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305573Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305584Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305578Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305601Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706161742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305516Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305554Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305555Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305559Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305616Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305540Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305561Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305564Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305591Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305488Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305531Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305520Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305522Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305532Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305544Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305542Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305543Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305548Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305549Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305551Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305552Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305558Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305560Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305565Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305563Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305572Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305585Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305575Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305582Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305607Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305614Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305580Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305538Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305557Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3706305571Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155425', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155343', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155302', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155245', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154814', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154774', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154734', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154049', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154014', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343140519', None, None, None, None, None, '2659516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 262 time used for this insertion : 4.457146167755127 save_final save missing photos in datou_result : time spend for datou_step_exec : 38.02576470375061 time spend to save output : 4.468536853790283 total time spend for step 2 : 42.494301557540894 step3:thcl Tue Mar 11 20:23:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.007041215896606445 time to convert the images to numpy array : 0.01936507225036621 time to import caffe and check if the image exist : 0.01197671890258789 time to convert the images to numpy array : 0.12370920181274414 time to import caffe and check if the image exist : 0.014910221099853516 time to convert the images to numpy array : 0.13637208938598633 time to import caffe and check if the image exist : 0.013556957244873047 time to convert the images to numpy array : 0.13918304443359375 time to import caffe and check if the image exist : 0.01933884620666504 time to convert the images to numpy array : 0.13551998138427734 time to import caffe and check if the image exist : 0.016056537628173828 time to convert the images to numpy array : 0.1394350528717041 time to import caffe and check if the image exist : 0.011713504791259766 time to convert the images to numpy array : 0.154008150100708 time to import caffe and check if the image exist : 0.01160430908203125 time to convert the images to numpy array : 0.16298842430114746 time to import caffe and check if the image exist : 0.01545858383178711 time to convert the images to numpy array : 0.16103124618530273 time to import caffe and check if the image exist : 0.009432315826416016 time to convert the images to numpy array : 0.17550396919250488 total time to convert the images to numpy array : 0.40841245651245117 list photo_ids error: [] list photo_ids correct : [-3706305538, -3706305557, -3706305571, -3706161742, -3706305516, -3706305512, -3706305525, -3706305554, -3706305555, -3706305559, -3706305616, -3706305481, -3706305573, -3706305584, -3706305578, -3706305601, -3706305606, -3706305600, -3706305595, -3706305612, -3706305611, -3706305531, -3706305520, -3706305518, -3706305522, -3706305521, -3706305526, -3706305517, -3706305532, -3706305534, -3706305497, -3706305492, -3706305488, -3706305486, -3706305494, -3706305487, -3706305511, -3706305500, -3706305508, -3706305607, -3706305610, -3706305609, -3706305608, -3706305614, -3706305477, -3706305507, -3706305580, -3706305587, -3706305544, -3706305542, -3706305543, -3706305548, -3706305549, -3706305551, -3706305552, -3706305558, -3706305560, -3706305475, -3706305498, -3706305499, -3706305502, -3706305545, -3706305550, -3706305546, -3706305562, -3706305577, -3706305476, -3706305479, -3706305540, -3706305561, -3706305564, -3706305591, -3706305615, -3706305482, -3706305480, -3706305565, -3706305563, -3706305572, -3706305592, -3706305586, -3706305585, -3706305575, -3706305582, -3706305598] number of photos to traite : 84 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10149 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 9930 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.82613205909729 time used to do the prediction : 0.19295334815979004 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 9928 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 9928 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.8686137199401855 time used to do the prediction : 0.1590564250946045 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.193450927734375e-05 save missing photos in datou_result : time spend for datou_step_exec : 9.325303792953491 time spend to save output : 0.0006124973297119141 total time spend for step 3 : 9.325916290283203 step4:merge_mask_thcl_custom Tue Mar 11 20:23:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 149 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1343155425, 1343155343, 1343155302, 1343155245, 1343154814, 1343154774, 1343154734, 1343154049, 1343154014, 1343140519] Looping around the photos to save general results len do output : 10 /1343155425Didn't retrieve data .Didn't retrieve data . /1343155343Didn't retrieve data .Didn't retrieve data . /1343155302Didn't retrieve data .Didn't retrieve data . /1343155245Didn't retrieve data .Didn't retrieve data . /1343154814Didn't retrieve data .Didn't retrieve data . /1343154774Didn't retrieve data .Didn't retrieve data . /1343154734Didn't retrieve data .Didn't retrieve data . /1343154049Didn't retrieve data .Didn't retrieve data . /1343154014Didn't retrieve data .Didn't retrieve data . /1343140519Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155425', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155343', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155302', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155245', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154814', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154774', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154734', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154049', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154014', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343140519', None, None, None, None, None, '2659516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.5609645843505859 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.20510029792785645 time spend to save output : 0.561579704284668 total time spend for step 4 : 0.7666800022125244 step5:rle_unique_nms_with_priority Tue Mar 11 20:23:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 11.115949630737305 time for calcul the mask position with numpy : 0.5839061737060547 nb_pixel_total : 3389271 time to create 1 rle with new method : 1.4670004844665527 time for calcul the mask position with numpy : 0.03646373748779297 nb_pixel_total : 37038 time to create 1 rle with old method : 0.04822897911071777 time for calcul the mask position with numpy : 0.04211926460266113 nb_pixel_total : 933 time to create 1 rle with old method : 0.001314401626586914 time for calcul the mask position with numpy : 0.03762197494506836 nb_pixel_total : 42209 time to create 1 rle with old method : 0.05018115043640137 time for calcul the mask position with numpy : 0.03711867332458496 nb_pixel_total : 9840 time to create 1 rle with old method : 0.011920928955078125 time for calcul the mask position with numpy : 0.02204751968383789 nb_pixel_total : 15750 time to create 1 rle with old method : 0.017748355865478516 time for calcul the mask position with numpy : 0.025647640228271484 nb_pixel_total : 162436 time to create 1 rle with new method : 0.42736124992370605 time for calcul the mask position with numpy : 0.023157358169555664 nb_pixel_total : 123986 time to create 1 rle with old method : 0.14056110382080078 time for calcul the mask position with numpy : 0.0224149227142334 nb_pixel_total : 36860 time to create 1 rle with old method : 0.04458951950073242 time for calcul the mask position with numpy : 0.02863025665283203 nb_pixel_total : 8265 time to create 1 rle with old method : 0.010361909866333008 time for calcul the mask position with numpy : 0.1912696361541748 nb_pixel_total : 3223652 time to create 1 rle with new method : 0.606339693069458 create new chi : 3.9614932537078857 time to delete rle : 0.01525115966796875 batch 1 Loaded 11 chid ids of type : 3726 Number RLEs to save : 14276 TO DO : save crop sub photo not yet done ! save time : 24.047532081604004 nb_obj : 15 nb_hashtags : 6 time to prepare the origin masks : 13.146669864654541 time for calcul the mask position with numpy : 0.9141147136688232 nb_pixel_total : 5932242 time to create 1 rle with new method : 1.0899848937988281 time for calcul the mask position with numpy : 0.02695775032043457 nb_pixel_total : 31140 time to create 1 rle with old method : 0.036824703216552734 time for calcul the mask position with numpy : 0.027400970458984375 nb_pixel_total : 88548 time to create 1 rle with old method : 0.10332298278808594 time for calcul the mask position with numpy : 0.022757768630981445 nb_pixel_total : 41619 time to create 1 rle with old method : 0.05520129203796387 time for calcul the mask position with numpy : 0.03127789497375488 nb_pixel_total : 16336 time to create 1 rle with old method : 0.028469562530517578 time for calcul the mask position with numpy : 0.029659271240234375 nb_pixel_total : 213998 time to create 1 rle with new method : 0.9244093894958496 time for calcul the mask position with numpy : 0.022213459014892578 nb_pixel_total : 9812 time to create 1 rle with old method : 0.010996103286743164 time for calcul the mask position with numpy : 0.022417068481445312 nb_pixel_total : 232633 time to create 1 rle with new method : 0.5442972183227539 time for calcul the mask position with numpy : 0.03472733497619629 nb_pixel_total : 22154 time to create 1 rle with old method : 0.025888681411743164 time for calcul the mask position with numpy : 0.03669452667236328 nb_pixel_total : 86412 time to create 1 rle with old method : 0.09800887107849121 time for calcul the mask position with numpy : 0.03488731384277344 nb_pixel_total : 35979 time to create 1 rle with old method : 0.04635047912597656 time for calcul the mask position with numpy : 0.04139256477355957 nb_pixel_total : 1606 time to create 1 rle with old method : 0.002560853958129883 time for calcul the mask position with numpy : 0.050246238708496094 nb_pixel_total : 121862 time to create 1 rle with old method : 0.19405841827392578 time for calcul the mask position with numpy : 0.03127789497375488 nb_pixel_total : 3168 time to create 1 rle with old method : 0.003484964370727539 time for calcul the mask position with numpy : 0.03142523765563965 nb_pixel_total : 201844 time to create 1 rle with new method : 0.5864365100860596 time for calcul the mask position with numpy : 0.031032085418701172 nb_pixel_total : 10887 time to create 1 rle with old method : 0.012348413467407227 create new chi : 5.265388011932373 time to delete rle : 0.001752614974975586 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 11343 TO DO : save crop sub photo not yet done ! save time : 5.181463003158569 nb_obj : 16 nb_hashtags : 6 time to prepare the origin masks : 14.58016586303711 time for calcul the mask position with numpy : 0.4414808750152588 nb_pixel_total : 6562614 time to create 1 rle with new method : 0.9566724300384521 time for calcul the mask position with numpy : 0.036467552185058594 nb_pixel_total : 133344 time to create 1 rle with old method : 0.15368103981018066 time for calcul the mask position with numpy : 0.03575944900512695 nb_pixel_total : 9605 time to create 1 rle with old method : 0.011085033416748047 time for calcul the mask position with numpy : 0.035275936126708984 nb_pixel_total : 37472 time to create 1 rle with old method : 0.04409193992614746 time for calcul the mask position with numpy : 0.03392839431762695 nb_pixel_total : 17423 time to create 1 rle with old method : 0.0193784236907959 time for calcul the mask position with numpy : 0.03410220146179199 nb_pixel_total : 1692 time to create 1 rle with old method : 0.0020093917846679688 time for calcul the mask position with numpy : 0.03438854217529297 nb_pixel_total : 58649 time to create 1 rle with old method : 0.08336400985717773 time for calcul the mask position with numpy : 0.028083324432373047 nb_pixel_total : 99954 time to create 1 rle with old method : 0.14026093482971191 time for calcul the mask position with numpy : 0.027924776077270508 nb_pixel_total : 12614 time to create 1 rle with old method : 0.017873764038085938 time for calcul the mask position with numpy : 0.02657032012939453 nb_pixel_total : 685 time to create 1 rle with old method : 0.0009243488311767578 time for calcul the mask position with numpy : 0.026968955993652344 nb_pixel_total : 37031 time to create 1 rle with old method : 0.058687686920166016 time for calcul the mask position with numpy : 0.04146003723144531 nb_pixel_total : 3390 time to create 1 rle with old method : 0.00525665283203125 time for calcul the mask position with numpy : 0.031800031661987305 nb_pixel_total : 10650 time to create 1 rle with old method : 0.01396632194519043 time for calcul the mask position with numpy : 0.02987360954284668 nb_pixel_total : 6267 time to create 1 rle with old method : 0.008406877517700195 time for calcul the mask position with numpy : 0.03142857551574707 nb_pixel_total : 3569 time to create 1 rle with old method : 0.004663228988647461 time for calcul the mask position with numpy : 0.031122684478759766 nb_pixel_total : 7227 time to create 1 rle with old method : 0.0169680118560791 time for calcul the mask position with numpy : 0.026882648468017578 nb_pixel_total : 48054 time to create 1 rle with old method : 0.05688285827636719 create new chi : 2.595339059829712 time to delete rle : 0.0009367465972900391 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 8989 TO DO : save crop sub photo not yet done ! save time : 12.665343523025513 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 16.71096634864807 time for calcul the mask position with numpy : 1.107170581817627 nb_pixel_total : 6598081 time to create 1 rle with new method : 0.9413516521453857 time for calcul the mask position with numpy : 0.04521775245666504 nb_pixel_total : 71685 time to create 1 rle with old method : 0.10060453414916992 time for calcul the mask position with numpy : 0.04211091995239258 nb_pixel_total : 19579 time to create 1 rle with old method : 0.02525615692138672 time for calcul the mask position with numpy : 0.048079490661621094 nb_pixel_total : 23565 time to create 1 rle with old method : 0.034673213958740234 time for calcul the mask position with numpy : 0.027215242385864258 nb_pixel_total : 10221 time to create 1 rle with old method : 0.011999130249023438 time for calcul the mask position with numpy : 0.024269819259643555 nb_pixel_total : 21358 time to create 1 rle with old method : 0.0255587100982666 time for calcul the mask position with numpy : 0.026023387908935547 nb_pixel_total : 1629 time to create 1 rle with old method : 0.0019555091857910156 time for calcul the mask position with numpy : 0.028703927993774414 nb_pixel_total : 36281 time to create 1 rle with old method : 0.042223215103149414 time for calcul the mask position with numpy : 0.026378631591796875 nb_pixel_total : 163074 time to create 1 rle with new method : 1.3885581493377686 time for calcul the mask position with numpy : 0.026549100875854492 nb_pixel_total : 2534 time to create 1 rle with old method : 0.00299835205078125 time for calcul the mask position with numpy : 0.023880481719970703 nb_pixel_total : 13040 time to create 1 rle with old method : 0.015041351318359375 time for calcul the mask position with numpy : 0.02378225326538086 nb_pixel_total : 17554 time to create 1 rle with old method : 0.025188207626342773 time for calcul the mask position with numpy : 0.027672290802001953 nb_pixel_total : 11306 time to create 1 rle with old method : 0.015096426010131836 time for calcul the mask position with numpy : 0.023693323135375977 nb_pixel_total : 24594 time to create 1 rle with old method : 0.03013443946838379 time for calcul the mask position with numpy : 0.023738384246826172 nb_pixel_total : 35739 time to create 1 rle with old method : 0.041271209716796875 create new chi : 4.320650577545166 time to delete rle : 0.0010082721710205078 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 9360 TO DO : save crop sub photo not yet done ! save time : 5.065328121185303 nb_obj : 11 nb_hashtags : 4 time to prepare the origin masks : 11.514583349227905 time for calcul the mask position with numpy : 1.5227947235107422 nb_pixel_total : 6502721 time to create 1 rle with new method : 0.8624937534332275 time for calcul the mask position with numpy : 0.05154895782470703 nb_pixel_total : 102628 time to create 1 rle with old method : 0.14085698127746582 time for calcul the mask position with numpy : 0.057881832122802734 nb_pixel_total : 36558 time to create 1 rle with old method : 0.08213162422180176 time for calcul the mask position with numpy : 0.04883837699890137 nb_pixel_total : 43362 time to create 1 rle with old method : 0.048662662506103516 time for calcul the mask position with numpy : 0.042627811431884766 nb_pixel_total : 1603 time to create 1 rle with old method : 0.0018992424011230469 time for calcul the mask position with numpy : 0.04596424102783203 nb_pixel_total : 51125 time to create 1 rle with old method : 0.06044435501098633 time for calcul the mask position with numpy : 0.05158185958862305 nb_pixel_total : 16385 time to create 1 rle with old method : 0.019934654235839844 time for calcul the mask position with numpy : 0.05128335952758789 nb_pixel_total : 136502 time to create 1 rle with old method : 0.1607801914215088 time for calcul the mask position with numpy : 0.049973487854003906 nb_pixel_total : 83609 time to create 1 rle with old method : 0.09635639190673828 time for calcul the mask position with numpy : 0.04574012756347656 nb_pixel_total : 2217 time to create 1 rle with old method : 0.00392603874206543 time for calcul the mask position with numpy : 0.029664993286132812 nb_pixel_total : 56734 time to create 1 rle with old method : 0.0764460563659668 time for calcul the mask position with numpy : 0.0277254581451416 nb_pixel_total : 16796 time to create 1 rle with old method : 0.018728971481323242 create new chi : 3.6498939990997314 time to delete rle : 0.0011522769927978516 batch 1 Loaded 12 chid ids of type : 3726 Number RLEs to save : 8958 TO DO : save crop sub photo not yet done ! save time : 8.004316091537476 nb_obj : 17 nb_hashtags : 7 time to prepare the origin masks : 12.14752745628357 time for calcul the mask position with numpy : 0.3897740840911865 nb_pixel_total : 6214863 time to create 1 rle with new method : 0.6202917098999023 time for calcul the mask position with numpy : 0.027581214904785156 nb_pixel_total : 111 time to create 1 rle with old method : 0.0003752708435058594 time for calcul the mask position with numpy : 0.025064468383789062 nb_pixel_total : 49521 time to create 1 rle with old method : 0.054247140884399414 time for calcul the mask position with numpy : 0.025027990341186523 nb_pixel_total : 6319 time to create 1 rle with old method : 0.007413148880004883 time for calcul the mask position with numpy : 0.024870872497558594 nb_pixel_total : 20604 time to create 1 rle with old method : 0.024091482162475586 time for calcul the mask position with numpy : 0.026866912841796875 nb_pixel_total : 59520 time to create 1 rle with old method : 0.0775594711303711 time for calcul the mask position with numpy : 0.03234553337097168 nb_pixel_total : 133290 time to create 1 rle with old method : 0.16857481002807617 time for calcul the mask position with numpy : 0.029682159423828125 nb_pixel_total : 3932 time to create 1 rle with old method : 0.005179643630981445 time for calcul the mask position with numpy : 0.029001474380493164 nb_pixel_total : 8541 time to create 1 rle with old method : 0.01078486442565918 time for calcul the mask position with numpy : 0.030389070510864258 nb_pixel_total : 70856 time to create 1 rle with old method : 0.09077620506286621 time for calcul the mask position with numpy : 0.02657032012939453 nb_pixel_total : 39650 time to create 1 rle with old method : 0.044319868087768555 time for calcul the mask position with numpy : 0.028094768524169922 nb_pixel_total : 15858 time to create 1 rle with old method : 0.01787114143371582 time for calcul the mask position with numpy : 0.026345252990722656 nb_pixel_total : 105645 time to create 1 rle with old method : 0.12620139122009277 time for calcul the mask position with numpy : 0.02978515625 nb_pixel_total : 63886 time to create 1 rle with old method : 0.07882094383239746 time for calcul the mask position with numpy : 0.029758214950561523 nb_pixel_total : 9574 time to create 1 rle with old method : 0.012720584869384766 time for calcul the mask position with numpy : 0.03376650810241699 nb_pixel_total : 25534 time to create 1 rle with old method : 0.032341957092285156 time for calcul the mask position with numpy : 0.029203176498413086 nb_pixel_total : 25177 time to create 1 rle with old method : 0.03216838836669922 time for calcul the mask position with numpy : 0.029407739639282227 nb_pixel_total : 197359 time to create 1 rle with new method : 0.7807705402374268 create new chi : 3.139383554458618 time to delete rle : 0.0017290115356445312 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 12751 TO DO : save crop sub photo not yet done ! save time : 14.833513021469116 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 10.387786626815796 time for calcul the mask position with numpy : 0.3726999759674072 nb_pixel_total : 6687040 time to create 1 rle with new method : 0.5600278377532959 time for calcul the mask position with numpy : 0.02330780029296875 nb_pixel_total : 14444 time to create 1 rle with old method : 0.018538236618041992 time for calcul the mask position with numpy : 0.022987842559814453 nb_pixel_total : 92918 time to create 1 rle with old method : 0.11696839332580566 time for calcul the mask position with numpy : 0.024747133255004883 nb_pixel_total : 2352 time to create 1 rle with old method : 0.0028314590454101562 time for calcul the mask position with numpy : 0.025538206100463867 nb_pixel_total : 23948 time to create 1 rle with old method : 0.029834270477294922 time for calcul the mask position with numpy : 0.0249025821685791 nb_pixel_total : 5002 time to create 1 rle with old method : 0.006496429443359375 time for calcul the mask position with numpy : 0.025829315185546875 nb_pixel_total : 4934 time to create 1 rle with old method : 0.006407976150512695 time for calcul the mask position with numpy : 0.024713993072509766 nb_pixel_total : 74390 time to create 1 rle with old method : 0.08819198608398438 time for calcul the mask position with numpy : 0.0404658317565918 nb_pixel_total : 52911 time to create 1 rle with old method : 0.07076478004455566 time for calcul the mask position with numpy : 0.02542567253112793 nb_pixel_total : 92301 time to create 1 rle with old method : 0.11496329307556152 create new chi : 1.7618262767791748 time to delete rle : 0.0008335113525390625 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 5724 TO DO : save crop sub photo not yet done ! save time : 4.568700075149536 nb_obj : 20 nb_hashtags : 6 time to prepare the origin masks : 17.39766526222229 time for calcul the mask position with numpy : 0.356431245803833 nb_pixel_total : 5875002 time to create 1 rle with new method : 0.670581579208374 time for calcul the mask position with numpy : 0.04668593406677246 nb_pixel_total : 255174 time to create 1 rle with new method : 0.8517856597900391 time for calcul the mask position with numpy : 0.0397186279296875 nb_pixel_total : 392 time to create 1 rle with old method : 0.0008411407470703125 time for calcul the mask position with numpy : 0.0402984619140625 nb_pixel_total : 370 time to create 1 rle with old method : 0.0007956027984619141 time for calcul the mask position with numpy : 0.040325164794921875 nb_pixel_total : 244 time to create 1 rle with old method : 0.00039458274841308594 time for calcul the mask position with numpy : 0.04127955436706543 nb_pixel_total : 85050 time to create 1 rle with old method : 0.09831619262695312 time for calcul the mask position with numpy : 0.04296469688415527 nb_pixel_total : 87630 time to create 1 rle with old method : 0.09900498390197754 time for calcul the mask position with numpy : 0.04201006889343262 nb_pixel_total : 8537 time to create 1 rle with old method : 0.009775876998901367 time for calcul the mask position with numpy : 0.04166889190673828 nb_pixel_total : 155739 time to create 1 rle with new method : 0.8968961238861084 time for calcul the mask position with numpy : 0.04161572456359863 nb_pixel_total : 29645 time to create 1 rle with old method : 0.03356432914733887 time for calcul the mask position with numpy : 0.041440725326538086 nb_pixel_total : 2124 time to create 1 rle with old method : 0.0025348663330078125 time for calcul the mask position with numpy : 0.04111170768737793 nb_pixel_total : 3967 time to create 1 rle with old method : 0.004651546478271484 time for calcul the mask position with numpy : 0.0393977165222168 nb_pixel_total : 14185 time to create 1 rle with old method : 0.0160067081451416 time for calcul the mask position with numpy : 0.029895305633544922 nb_pixel_total : 145233 time to create 1 rle with old method : 0.1562209129333496 time for calcul the mask position with numpy : 0.026464462280273438 nb_pixel_total : 14691 time to create 1 rle with old method : 0.0158233642578125 time for calcul the mask position with numpy : 0.025431156158447266 nb_pixel_total : 16192 time to create 1 rle with old method : 0.01790142059326172 time for calcul the mask position with numpy : 0.027549028396606445 nb_pixel_total : 20120 time to create 1 rle with old method : 0.022722721099853516 time for calcul the mask position with numpy : 0.027149677276611328 nb_pixel_total : 52243 time to create 1 rle with old method : 0.05701613426208496 time for calcul the mask position with numpy : 0.028223037719726562 nb_pixel_total : 101479 time to create 1 rle with old method : 0.11104321479797363 time for calcul the mask position with numpy : 0.02881908416748047 nb_pixel_total : 75026 time to create 1 rle with old method : 0.08120393753051758 time for calcul the mask position with numpy : 0.028118371963500977 nb_pixel_total : 107197 time to create 1 rle with old method : 0.1220388412475586 create new chi : 4.458819389343262 time to delete rle : 0.001516103744506836 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 12528 TO DO : save crop sub photo not yet done ! save time : 5.786375999450684 nb_obj : 22 nb_hashtags : 6 time to prepare the origin masks : 16.71476697921753 time for calcul the mask position with numpy : 0.8515186309814453 nb_pixel_total : 5878921 time to create 1 rle with new method : 0.4269583225250244 time for calcul the mask position with numpy : 0.035112857818603516 nb_pixel_total : 24960 time to create 1 rle with old method : 0.028151750564575195 time for calcul the mask position with numpy : 0.03437399864196777 nb_pixel_total : 16999 time to create 1 rle with old method : 0.019165515899658203 time for calcul the mask position with numpy : 0.03598380088806152 nb_pixel_total : 152347 time to create 1 rle with new method : 0.5614562034606934 time for calcul the mask position with numpy : 0.034819841384887695 nb_pixel_total : 25298 time to create 1 rle with old method : 0.028111696243286133 time for calcul the mask position with numpy : 0.03293561935424805 nb_pixel_total : 13922 time to create 1 rle with old method : 0.016742467880249023 time for calcul the mask position with numpy : 0.03157997131347656 nb_pixel_total : 72246 time to create 1 rle with old method : 0.08355951309204102 time for calcul the mask position with numpy : 0.022958040237426758 nb_pixel_total : 88678 time to create 1 rle with old method : 0.09993147850036621 time for calcul the mask position with numpy : 0.020796775817871094 nb_pixel_total : 978 time to create 1 rle with old method : 0.0011327266693115234 time for calcul the mask position with numpy : 0.022704601287841797 nb_pixel_total : 32173 time to create 1 rle with old method : 0.036160945892333984 time for calcul the mask position with numpy : 0.022735595703125 nb_pixel_total : 95903 time to create 1 rle with old method : 0.10696697235107422 time for calcul the mask position with numpy : 0.022312641143798828 nb_pixel_total : 91747 time to create 1 rle with old method : 0.1018209457397461 time for calcul the mask position with numpy : 0.021633148193359375 nb_pixel_total : 42437 time to create 1 rle with old method : 0.04770016670227051 time for calcul the mask position with numpy : 0.022248029708862305 nb_pixel_total : 2396 time to create 1 rle with old method : 0.002685546875 time for calcul the mask position with numpy : 0.02145218849182129 nb_pixel_total : 1908 time to create 1 rle with old method : 0.0023152828216552734 time for calcul the mask position with numpy : 0.021965503692626953 nb_pixel_total : 1319 time to create 1 rle with old method : 0.0015549659729003906 time for calcul the mask position with numpy : 0.02202773094177246 nb_pixel_total : 64185 time to create 1 rle with old method : 0.07211780548095703 time for calcul the mask position with numpy : 0.021843671798706055 nb_pixel_total : 13996 time to create 1 rle with old method : 0.015976905822753906 time for calcul the mask position with numpy : 0.020833730697631836 nb_pixel_total : 2005 time to create 1 rle with old method : 0.0023190975189208984 time for calcul the mask position with numpy : 0.021845340728759766 nb_pixel_total : 42028 time to create 1 rle with old method : 0.07900500297546387 time for calcul the mask position with numpy : 0.024458646774291992 nb_pixel_total : 278451 time to create 1 rle with new method : 0.6474106311798096 time for calcul the mask position with numpy : 0.022328615188598633 nb_pixel_total : 42942 time to create 1 rle with old method : 0.05121612548828125 time for calcul the mask position with numpy : 0.026745080947875977 nb_pixel_total : 64401 time to create 1 rle with old method : 0.07422280311584473 create new chi : 4.0071327686309814 time to delete rle : 0.0016624927520751953 batch 1 Loaded 23 chid ids of type : 3726 Number RLEs to save : 15305 TO DO : save crop sub photo not yet done ! save time : 14.538971185684204 nb_obj : 2 nb_hashtags : 2 time to prepare the origin masks : 3.4513070583343506 time for calcul the mask position with numpy : 0.034578561782836914 nb_pixel_total : 2007004 time to create 1 rle with new method : 0.8711333274841309 time for calcul the mask position with numpy : 0.02517867088317871 nb_pixel_total : 42512 time to create 1 rle with old method : 0.05269265174865723 time for calcul the mask position with numpy : 0.3940892219543457 nb_pixel_total : 5000724 time to create 1 rle with new method : 0.8410050868988037 create new chi : 2.29510498046875 time to delete rle : 0.0012395381927490234 batch 1 Loaded 3 chid ids of type : 3726 Number RLEs to save : 9374 TO DO : save crop sub photo not yet done ! save time : 3.9960052967071533 map_output_result : {1343155425: (0.0, 'Should be the crop_list due to order', 0), 1343155343: (0.0, 'Should be the crop_list due to order', 0), 1343155302: (0.0, 'Should be the crop_list due to order', 0), 1343155245: (0.0, 'Should be the crop_list due to order', 0), 1343154814: (0.0, 'Should be the crop_list due to order', 0), 1343154774: (0.0, 'Should be the crop_list due to order', 0), 1343154734: (0.0, 'Should be the crop_list due to order', 0), 1343154049: (0.0, 'Should be the crop_list due to order', 0), 1343154014: (0.0, 'Should be the crop_list due to order', 0), 1343140519: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1343155425, 1343155343, 1343155302, 1343155245, 1343154814, 1343154774, 1343154734, 1343154049, 1343154014, 1343140519] Looping around the photos to save general results len do output : 10 /1343155425.Didn't retrieve data . /1343155343.Didn't retrieve data . /1343155302.Didn't retrieve data . /1343155245.Didn't retrieve data . /1343154814.Didn't retrieve data . /1343154774.Didn't retrieve data . /1343154734.Didn't retrieve data . /1343154049.Didn't retrieve data . /1343154014.Didn't retrieve data . /1343140519.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155425', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155343', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155302', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155245', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154814', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154774', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154734', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154049', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154014', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343140519', None, None, None, None, None, '2659516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.020219802856445312 save_final save missing photos in datou_result : time spend for datou_step_exec : 262.41879415512085 time spend to save output : 0.025908470153808594 total time spend for step 5 : 262.44470262527466 step6:crop_condition Tue Mar 11 20:28:04 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 146 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741721289_2576476 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.4272289276123047 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741721294_2576476 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.1606130599975586 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741721298_2576476 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.559246301651001 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741721302_2576476 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.6218750476837158 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741721306_2576476 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 6.113152503967285 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741721315_2576476 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.8382649421691895 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1741721317_2576476 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.6953074932098389 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1343155425, 1343155343, 1343155302, 1343155245, 1343154814, 1343154774, 1343154734, 1343154049, 1343154014, 1343140519] Looping around the photos to save general results len do output : 28 /1343185470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185486Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185530Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185587Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185595Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185694Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185708Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185709Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1343185724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155425', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155343', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155302', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155245', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154814', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154774', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154734', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154049', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154014', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343140519', None, None, None, None, None, '2659516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 94 time used for this insertion : 0.030939102172851562 save_final save missing photos in datou_result : time spend for datou_step_exec : 49.3879554271698 time spend to save output : 0.06094980239868164 total time spend for step 6 : 49.44890522956848 step7:ventilate_hashtags_in_portfolio Tue Mar 11 20:28:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 21307714 get user id for portfolio 21307714 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21307714 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','flou','Carton_gris','plastique','cartonnette','Carton_brun','environnement','mal_croppe','Teint_Dans_La_Masse','papier','autre_refus','kraft')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21307714 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','flou','Carton_gris','plastique','cartonnette','Carton_brun','environnement','mal_croppe','Teint_Dans_La_Masse','papier','autre_refus','kraft')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21307714 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','flou','Carton_gris','plastique','cartonnette','Carton_brun','environnement','mal_croppe','Teint_Dans_La_Masse','papier','autre_refus','kraft')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/21308295,21308296,21308297,21308298,21308299,21308300,21308301,21308302,21308303,21308304,21308305,21308306?tags=metal,flou,Carton_gris,plastique,cartonnette,Carton_brun,environnement,mal_croppe,Teint_Dans_La_Masse,papier,autre_refus,kraft Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1343155425, 1343155343, 1343155302, 1343155245, 1343154814, 1343154774, 1343154734, 1343154049, 1343154014, 1343140519] Looping around the photos to save general results len do output : 1 /21307714. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155425', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155343', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155302', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155245', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154814', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154774', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154734', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154049', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154014', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343140519', None, None, None, None, None, '2659516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014075279235839844 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.9320273399353027 time spend to save output : 0.014686822891235352 total time spend for step 7 : 1.946714162826538 step8:final Tue Mar 11 20:28:55 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1343155425: ('0.07765977334100109',), 1343155343: ('0.07765977334100109',), 1343155302: ('0.07765977334100109',), 1343155245: ('0.07765977334100109',), 1343154814: ('0.07765977334100109',), 1343154774: ('0.07765977334100109',), 1343154734: ('0.07765977334100109',), 1343154049: ('0.07765977334100109',), 1343154014: ('0.07765977334100109',), 1343140519: ('0.07765977334100109',)} new output for save of step final : {1343155425: ('0.07765977334100109',), 1343155343: ('0.07765977334100109',), 1343155302: ('0.07765977334100109',), 1343155245: ('0.07765977334100109',), 1343154814: ('0.07765977334100109',), 1343154774: ('0.07765977334100109',), 1343154734: ('0.07765977334100109',), 1343154049: ('0.07765977334100109',), 1343154014: ('0.07765977334100109',), 1343140519: ('0.07765977334100109',)} [1343155425, 1343155343, 1343155302, 1343155245, 1343154814, 1343154774, 1343154734, 1343154049, 1343154014, 1343140519] Looping around the photos to save general results len do output : 10 /1343155425.Didn't retrieve data . /1343155343.Didn't retrieve data . /1343155302.Didn't retrieve data . /1343155245.Didn't retrieve data . /1343154814.Didn't retrieve data . /1343154774.Didn't retrieve data . /1343154734.Didn't retrieve data . /1343154049.Didn't retrieve data . /1343154014.Didn't retrieve data . /1343140519.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155425', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155343', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155302', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155245', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154814', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154774', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154734', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154049', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154014', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343140519', None, None, None, None, None, '2659516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013774633407592773 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.27410364151000977 time spend to save output : 0.014294862747192383 total time spend for step 8 : 0.28839850425720215 step9:velours_tree Tue Mar 11 20:28:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.2170426845550537 time spend to save output : 9.393692016601562e-05 total time spend for step 9 : 0.21713662147521973 step10:send_mail_cod Tue Mar 11 20:28:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P21307714_11-03-2025_20_28_56.pdf 21308295 imagette213082951741721336 21308296 imagette213082961741721336 21308298 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette213082981741721336 21308299 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette213082991741721337 21308302 imagette213083021741721337 21308305 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette213083051741721338 21308306 change filename to text .change filename to text .imagette213083061741721339 21308297 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette213082971741721339 21308300 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette213083001741721340 21308303 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette213083031741721343 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=21307714 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/21308295,21308296,21308297,21308298,21308299,21308300,21308301,21308302,21308303,21308304,21308305,21308306?tags=metal,flou,Carton_gris,plastique,cartonnette,Carton_brun,environnement,mal_croppe,Teint_Dans_La_Masse,papier,autre_refus,kraft your option no_mail is active, we will not send the real mail to your client args[1343155425] : ('0.07765977334100109',) no score found for photo 1343155425 We are sending mail with results at report@fotonower.com args[1343155343] : ('0.07765977334100109',) no score found for photo 1343155343 We are sending mail with results at report@fotonower.com args[1343155302] : ('0.07765977334100109',) no score found for photo 1343155302 We are sending mail with results at report@fotonower.com args[1343155245] : ('0.07765977334100109',) no score found for photo 1343155245 We are sending mail with results at report@fotonower.com args[1343154814] : ('0.07765977334100109',) no score found for photo 1343154814 We are sending mail with results at report@fotonower.com args[1343154774] : ('0.07765977334100109',) no score found for photo 1343154774 We are sending mail with results at report@fotonower.com args[1343154734] : ('0.07765977334100109',) no score found for photo 1343154734 We are sending mail with results at report@fotonower.com args[1343154049] : ('0.07765977334100109',) no score found for photo 1343154049 We are sending mail with results at report@fotonower.com args[1343154014] : ('0.07765977334100109',) no score found for photo 1343154014 We are sending mail with results at report@fotonower.com args[1343140519] : ('0.07765977334100109',) no score found for photo 1343140519 We are sending mail with results at report@fotonower.com refus_total : 0.07765977334100109 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=21307714 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1343140456,1343154814,1343154774,1343154049,1343154014,1343155245,1343155302,1343155343,1343140080,1343140413,1343140365,1343140197,1343140167,1343140121,1343140093,1343140214,1343154734,1343155425,1343140519,1343140512) Found this number of photos: 20 begin to download photo : 1343140456 begin to download photo : 1343155245 begin to download photo : 1343140365 begin to download photo : 1343140214 download finish for photo 1343140456 begin to download photo : 1343154814 download finish for photo 1343155245 begin to download photo : 1343155302 download finish for photo 1343140365 begin to download photo : 1343140197 download finish for photo 1343140214 begin to download photo : 1343154734 download finish for photo 1343154814 begin to download photo : 1343154774 download finish for photo 1343154734 begin to download photo : 1343155425 download finish for photo 1343140197 begin to download photo : 1343140167 download finish for photo 1343155302 begin to download photo : 1343155343 download finish for photo 1343155425 begin to download photo : 1343140519 download finish for photo 1343154774 begin to download photo : 1343154049 download finish for photo 1343140167 begin to download photo : 1343140121 download finish for photo 1343155343 begin to download photo : 1343140080 download finish for photo 1343140519 begin to download photo : 1343140512 download finish for photo 1343140121 begin to download photo : 1343140093 download finish for photo 1343154049 begin to download photo : 1343154014 download finish for photo 1343140080 begin to download photo : 1343140413 download finish for photo 1343140093 download finish for photo 1343154014 download finish for photo 1343140512 download finish for photo 1343140413 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21307714_11-03-2025_20_28_56.pdf results_Auto_P21307714_11-03-2025_20_28_56.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21307714_11-03-2025_20_28_56.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','21307714','results_Auto_P21307714_11-03-2025_20_28_56.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21307714_11-03-2025_20_28_56.pdf','pdf','','1.1','0.07765977334100109') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1343155425, 1343155343, 1343155302, 1343155245, 1343154814, 1343154774, 1343154734, 1343154049, 1343154014, 1343140519] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155425', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155343', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155302', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155245', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154814', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154774', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154734', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154049', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154014', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343140519', None, None, None, None, None, '2659516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 2.365426778793335 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.83246111869812 time spend to save output : 2.365687131881714 total time spend for step 10 : 15.198148250579834 step11:split_time_score Tue Mar 11 20:29:11 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('18', 22),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 11032025 21307714 Nombre de photos uploadées : 22 / 23040 (0%) 11032025 21307714 Nombre de photos taguées (types de déchets): 0 / 22 (0%) 11032025 21307714 Nombre de photos taguées (volume) : 0 / 22 (0%) elapsed_time : load_data_split_time_score 2.6226043701171875e-06 elapsed_time : order_list_meta_photo_and_scores 1.4543533325195312e-05 ?????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0014028549194335938 elapsed_time : insert_dashboard_record_day_entry 0.02187514305114746 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.05881318002080126 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291426_11-03-2025_16_06_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291426 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291426 AND mptpi.`type`=4207 To do Qualite : 0.07848604537653218 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268995_11-03-2025_10_04_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268995 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268995 AND mptpi.`type`=4207 To do Qualite : 0.21508235889842053 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268996_11-03-2025_09_21_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268996 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268996 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268997_11-03-2025_09_36_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268997 order by id desc limit 1 Qualite : 0.2996323014615577 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21268998_11-03-2025_09_21_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21268998 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21268998 AND mptpi.`type`=4200 To do Qualite : 0.059107923268759154 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21269000_11-03-2025_09_44_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21269000 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21269000 AND mptpi.`type`=4209 To do Qualite : 0.09529821171173529 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21269001_11-03-2025_09_17_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21269001 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21269001 AND mptpi.`type`=3726 To do Qualite : 0.12370809963529884 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21270945_11-03-2025_12_54_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21270945 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21270945 AND mptpi.`type`=3327 To do Qualite : 0.044100679011939585 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21276914_11-03-2025_11_38_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21276914 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21276914 AND mptpi.`type`=4207 To do Qualite : 0.04798165055335807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21276915_11-03-2025_12_41_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21276915 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21276915 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290022 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290047 order by id desc limit 1 Qualite : 0.24539234952758107 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282296_11-03-2025_13_26_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282296 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282296 AND mptpi.`type`=3594 To do Qualite : 0.23651912322606505 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282297_11-03-2025_14_28_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282297 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282297 AND mptpi.`type`=3594 To do Qualite : 0.30571381203013953 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282298_11-03-2025_13_24_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282298 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282298 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282299_11-03-2025_12_43_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282299 order by id desc limit 1 Qualite : 0.3846979317267506 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282300_11-03-2025_13_27_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282300 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282300 AND mptpi.`type`=4200 To do Qualite : 0.3844764824714978 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282301_11-03-2025_13_46_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282301 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21282301 AND mptpi.`type`=4200 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21282303_11-03-2025_12_44_04.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21282303 order by id desc limit 1 Qualite : 0.21547889555127603 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290048_11-03-2025_15_09_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290048 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290048 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290049 order by id desc limit 1 Qualite : 0.056030119320647845 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290050_11-03-2025_15_43_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290050 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11488 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11496 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11497 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11495 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11495 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11489 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11575 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11575 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11491 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11490 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11490 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11498 send_mail_cod have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Step 11499 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11491 doesn't seem to be define in the database( WARNING : type of input 3 of step 11490 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11488 doesn't seem to be define in the database( WARNING : type of input 2 of step 11492 doesn't seem to be define in the database( WARNING : output 1 of step 11488 have datatype=2 whereas input 1 of step 11495 have datatype=7 WARNING : type of output 2 of step 11495 doesn't seem to be define in the database( WARNING : type of input 1 of step 11489 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11491 have datatype=10 whereas input 3 of step 11498 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11575 doesn't seem to be define in the database( WARNING : output 1 of step 11489 have datatype=7 whereas input 2 of step 11575 have datatype=None WARNING : type of output 3 of step 11575 doesn't seem to be define in the database( WARNING : type of input 1 of step 11491 doesn't seem to be define in the database( WARNING : output 0 of step 11491 have datatype=10 whereas input 0 of step 11581 have datatype=18 WARNING : type of input 5 of step 11498 doesn't seem to be define in the database( WARNING : output 0 of step 11581 have datatype=11 whereas input 5 of step 11498 have datatype=None WARNING : type of output 1 of step 11496 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : type of output 1 of step 11497 doesn't seem to be define in the database( WARNING : type of input 3 of step 11492 doesn't seem to be define in the database( WARNING : output 0 of step 11495 have datatype=1 whereas input 0 of step 11489 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290050 AND mptpi.`type`=4209 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290051 order by id desc limit 1 Qualite : 0.04833780284360806 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21290052_11-03-2025_15_05_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21290052 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21290052 AND mptpi.`type`=4207 To do Qualite : 0.0812687785847433 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291428_11-03-2025_15_38_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291428 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291428 AND mptpi.`type`=3726 To do Qualite : 0.07647240353513372 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21291429_11-03-2025_16_22_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21291429 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21291429 AND mptpi.`type`=4211 To do Qualite : 0.11562154222841053 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21299945_11-03-2025_18_24_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21299945 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21299945 AND mptpi.`type`=3327 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21297107_11-03-2025_17_13_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21297107 order by id desc limit 1 Qualite : 0.0 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21297109_11-03-2025_18_02_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21297109 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21297109 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21299946_11-03-2025_19_02_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21299946 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21304676 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21304677 order by id desc limit 1 Qualite : 0.053237275430592125 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21304678_11-03-2025_19_43_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21304678 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21304678 AND mptpi.`type`=4211 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21304679_11-03-2025_19_33_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21304679 order by id desc limit 1 Qualite : 0.24620547984497312 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21304704 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21304704 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21307710 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21307712 order by id desc limit 1 Qualite : 0.07765977334100109 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21307714_11-03-2025_20_28_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21307714 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21307714 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21307715 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21307716 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21307717 order by id desc limit 1 Qualite : 0.07363416515780094 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P21307718_11-03-2025_20_28_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21307718 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21307718 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 21307731 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'11032025': {'nb_upload': 22, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1343155425, 1343155343, 1343155302, 1343155245, 1343154814, 1343154774, 1343154734, 1343154049, 1343154014, 1343140519] Looping around the photos to save general results len do output : 1 /21307714Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155425', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155343', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155302', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343155245', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154814', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154774', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154734', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154049', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343154014', None, None, None, None, None, '2659516') ('3459', None, None, None, None, None, None, None, '2659516') ('3459', '21307714', '1343140519', None, None, None, None, None, '2659516') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.01566910743713379 save_final save missing photos in datou_result : time spend for datou_step_exec : 15.211443662643433 time spend to save output : 0.015944242477416992 total time spend for step 11 : 15.22738790512085 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 143.54user 173.75system 9:04.52elapsed 58%CPU (0avgtext+0avgdata 4425316maxresident)k 338856inputs+128976outputs (21681major+12438861minor)pagefaults 0swaps