python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 855890 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 26>10 please execute split_portfolio.py -i 20425795 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2574456'] with mtr_portfolio_ids : ['20425795'] and first list_photo_ids : [] new path : /proc/855890/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.9118726253509521 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Feb 11 03:28:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-11 03:28:32.570002: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-11 03:28:32.603131: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-11 03:28:32.604665: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f5b84000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-11 03:28:32.604720: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-11 03:28:32.607918: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-11 03:28:32.863324: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x15157360 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-11 03:28:32.863365: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-11 03:28:32.864523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-11 03:28:32.865616: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-11 03:28:32.878445: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-11 03:28:32.895494: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-11 03:28:32.897555: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-11 03:28:32.931950: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-11 03:28:32.937988: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-11 03:28:32.988934: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-11 03:28:32.991495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-11 03:28:32.991620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-11 03:28:32.992980: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-11 03:28:32.993002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-11 03:28:32.993014: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-11 03:28:32.995316: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9985 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-11 03:28:33.343701: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-11 03:28:33.343797: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-11 03:28:33.343820: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-11 03:28:33.343843: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-11 03:28:33.343861: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-11 03:28:33.343878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-11 03:28:33.343896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-11 03:28:33.343914: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-11 03:28:33.345478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-11 03:28:33.346806: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-11 03:28:33.346840: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-11 03:28:33.346859: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-11 03:28:33.346876: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-11 03:28:33.346892: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-11 03:28:33.346938: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-11 03:28:33.346956: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-11 03:28:33.346973: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-11 03:28:33.348423: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-11 03:28:33.348449: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-11 03:28:33.348458: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-11 03:28:33.348466: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-11 03:28:33.349797: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9985 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-11 03:28:45.094404: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-11 03:28:45.305286: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 14 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 2 Detection mask done ! Trying to reset tf kernel 856151 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5485 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10774 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0019800662994384766 nb_pixel_total : 79103 time to create 1 rle with old method : 0.08956670761108398 length of segment : 355 time for calcul the mask position with numpy : 0.00023055076599121094 nb_pixel_total : 12338 time to create 1 rle with old method : 0.014682769775390625 length of segment : 82 time for calcul the mask position with numpy : 0.00032067298889160156 nb_pixel_total : 6608 time to create 1 rle with old method : 0.008291006088256836 length of segment : 90 time for calcul the mask position with numpy : 0.0005500316619873047 nb_pixel_total : 18179 time to create 1 rle with old method : 0.026613712310791016 length of segment : 259 time for calcul the mask position with numpy : 0.00012350082397460938 nb_pixel_total : 3385 time to create 1 rle with old method : 0.004282951354980469 length of segment : 111 time for calcul the mask position with numpy : 0.00038313865661621094 nb_pixel_total : 16187 time to create 1 rle with old method : 0.019741296768188477 length of segment : 282 time for calcul the mask position with numpy : 0.0010004043579101562 nb_pixel_total : 55601 time to create 1 rle with old method : 0.06322622299194336 length of segment : 339 time for calcul the mask position with numpy : 0.0010437965393066406 nb_pixel_total : 59496 time to create 1 rle with old method : 0.06819510459899902 length of segment : 339 time for calcul the mask position with numpy : 9.274482727050781e-05 nb_pixel_total : 2470 time to create 1 rle with old method : 0.002966642379760742 length of segment : 79 time for calcul the mask position with numpy : 0.0017313957214355469 nb_pixel_total : 49070 time to create 1 rle with old method : 0.05699276924133301 length of segment : 780 time for calcul the mask position with numpy : 0.0006542205810546875 nb_pixel_total : 27740 time to create 1 rle with old method : 0.03177905082702637 length of segment : 452 time for calcul the mask position with numpy : 0.0005888938903808594 nb_pixel_total : 767 time to create 1 rle with old method : 0.001085042953491211 length of segment : 60 time for calcul the mask position with numpy : 0.0027294158935546875 nb_pixel_total : 162076 time to create 1 rle with new method : 0.010473966598510742 length of segment : 359 time for calcul the mask position with numpy : 0.00012040138244628906 nb_pixel_total : 3077 time to create 1 rle with old method : 0.0038635730743408203 length of segment : 108 time for calcul the mask position with numpy : 0.0005192756652832031 nb_pixel_total : 11102 time to create 1 rle with old method : 0.013623476028442383 length of segment : 210 time for calcul the mask position with numpy : 0.0005373954772949219 nb_pixel_total : 30768 time to create 1 rle with old method : 0.035585641860961914 length of segment : 239 time for calcul the mask position with numpy : 0.00019478797912597656 nb_pixel_total : 4972 time to create 1 rle with old method : 0.0061419010162353516 length of segment : 130 time for calcul the mask position with numpy : 0.0008528232574462891 nb_pixel_total : 52497 time to create 1 rle with old method : 0.06499028205871582 length of segment : 190 time for calcul the mask position with numpy : 0.0003502368927001953 nb_pixel_total : 17006 time to create 1 rle with old method : 0.020012855529785156 length of segment : 173 time for calcul the mask position with numpy : 0.003223419189453125 nb_pixel_total : 144636 time to create 1 rle with old method : 0.1619889736175537 length of segment : 601 time for calcul the mask position with numpy : 0.0035195350646972656 nb_pixel_total : 190527 time to create 1 rle with new method : 0.01241302490234375 length of segment : 652 time for calcul the mask position with numpy : 0.0011949539184570312 nb_pixel_total : 66831 time to create 1 rle with old method : 0.07508349418640137 length of segment : 490 time for calcul the mask position with numpy : 0.0008900165557861328 nb_pixel_total : 49669 time to create 1 rle with old method : 0.05742239952087402 length of segment : 309 time for calcul the mask position with numpy : 0.0019428730010986328 nb_pixel_total : 125446 time to create 1 rle with old method : 0.14085936546325684 length of segment : 521 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 995 time to create 1 rle with old method : 0.001277923583984375 length of segment : 54 time for calcul the mask position with numpy : 0.0017728805541992188 nb_pixel_total : 85282 time to create 1 rle with old method : 0.09793519973754883 length of segment : 363 time for calcul the mask position with numpy : 0.0003657341003417969 nb_pixel_total : 19261 time to create 1 rle with old method : 0.022884130477905273 length of segment : 144 time for calcul the mask position with numpy : 0.0006303787231445312 nb_pixel_total : 39621 time to create 1 rle with old method : 0.046646833419799805 length of segment : 152 time for calcul the mask position with numpy : 0.006127834320068359 nb_pixel_total : 207975 time to create 1 rle with new method : 0.01253056526184082 length of segment : 1032 time for calcul the mask position with numpy : 0.0022640228271484375 nb_pixel_total : 157209 time to create 1 rle with new method : 0.005452871322631836 length of segment : 604 time for calcul the mask position with numpy : 9.632110595703125e-05 nb_pixel_total : 2913 time to create 1 rle with old method : 0.0036802291870117188 length of segment : 72 time for calcul the mask position with numpy : 7.343292236328125e-05 nb_pixel_total : 2209 time to create 1 rle with old method : 0.002983570098876953 length of segment : 39 time for calcul the mask position with numpy : 0.002897024154663086 nb_pixel_total : 176892 time to create 1 rle with new method : 0.00719904899597168 length of segment : 938 time for calcul the mask position with numpy : 0.0002040863037109375 nb_pixel_total : 8614 time to create 1 rle with old method : 0.010258674621582031 length of segment : 153 time for calcul the mask position with numpy : 0.00043082237243652344 nb_pixel_total : 19294 time to create 1 rle with old method : 0.02257061004638672 length of segment : 231 time for calcul the mask position with numpy : 0.0005462169647216797 nb_pixel_total : 32794 time to create 1 rle with old method : 0.03702974319458008 length of segment : 278 time for calcul the mask position with numpy : 0.00018453598022460938 nb_pixel_total : 10041 time to create 1 rle with old method : 0.012132644653320312 length of segment : 84 time for calcul the mask position with numpy : 0.0006508827209472656 nb_pixel_total : 29920 time to create 1 rle with old method : 0.0457003116607666 length of segment : 303 time for calcul the mask position with numpy : 0.0002968311309814453 nb_pixel_total : 9828 time to create 1 rle with old method : 0.012115716934204102 length of segment : 188 time for calcul the mask position with numpy : 0.0019345283508300781 nb_pixel_total : 111593 time to create 1 rle with old method : 0.1280214786529541 length of segment : 589 time for calcul the mask position with numpy : 0.00042057037353515625 nb_pixel_total : 16876 time to create 1 rle with old method : 0.0204007625579834 length of segment : 270 time for calcul the mask position with numpy : 0.0021898746490478516 nb_pixel_total : 147939 time to create 1 rle with old method : 0.16627192497253418 length of segment : 591 time for calcul the mask position with numpy : 0.0018579959869384766 nb_pixel_total : 100304 time to create 1 rle with old method : 0.11250996589660645 length of segment : 464 time for calcul the mask position with numpy : 0.0006470680236816406 nb_pixel_total : 21276 time to create 1 rle with old method : 0.025609493255615234 length of segment : 304 time for calcul the mask position with numpy : 0.00014448165893554688 nb_pixel_total : 4539 time to create 1 rle with old method : 0.007693767547607422 length of segment : 96 time for calcul the mask position with numpy : 0.0024666786193847656 nb_pixel_total : 155124 time to create 1 rle with new method : 0.0056247711181640625 length of segment : 664 time for calcul the mask position with numpy : 0.0014705657958984375 nb_pixel_total : 91081 time to create 1 rle with old method : 0.10317277908325195 length of segment : 385 time for calcul the mask position with numpy : 0.0014355182647705078 nb_pixel_total : 78448 time to create 1 rle with old method : 0.0913698673248291 length of segment : 290 time for calcul the mask position with numpy : 0.00015115737915039062 nb_pixel_total : 5789 time to create 1 rle with old method : 0.007892131805419922 length of segment : 78 time for calcul the mask position with numpy : 0.0007469654083251953 nb_pixel_total : 38175 time to create 1 rle with old method : 0.043120384216308594 length of segment : 389 time for calcul the mask position with numpy : 0.0003561973571777344 nb_pixel_total : 18909 time to create 1 rle with old method : 0.022175073623657227 length of segment : 172 time for calcul the mask position with numpy : 0.00013303756713867188 nb_pixel_total : 4637 time to create 1 rle with old method : 0.005996227264404297 length of segment : 97 time for calcul the mask position with numpy : 7.367134094238281e-05 nb_pixel_total : 1170 time to create 1 rle with old method : 0.001520395278930664 length of segment : 57 time for calcul the mask position with numpy : 9.179115295410156e-05 nb_pixel_total : 3324 time to create 1 rle with old method : 0.004217863082885742 length of segment : 75 time for calcul the mask position with numpy : 0.0019915103912353516 nb_pixel_total : 126849 time to create 1 rle with old method : 0.14379072189331055 length of segment : 756 time for calcul the mask position with numpy : 0.00034928321838378906 nb_pixel_total : 12177 time to create 1 rle with old method : 0.014844655990600586 length of segment : 149 time for calcul the mask position with numpy : 0.0020825862884521484 nb_pixel_total : 73103 time to create 1 rle with old method : 0.08360004425048828 length of segment : 456 time for calcul the mask position with numpy : 0.0004744529724121094 nb_pixel_total : 24823 time to create 1 rle with old method : 0.029439687728881836 length of segment : 136 time for calcul the mask position with numpy : 0.00035858154296875 nb_pixel_total : 13671 time to create 1 rle with old method : 0.01609492301940918 length of segment : 164 time for calcul the mask position with numpy : 0.001596212387084961 nb_pixel_total : 99376 time to create 1 rle with old method : 0.11376023292541504 length of segment : 430 time for calcul the mask position with numpy : 0.003199338912963867 nb_pixel_total : 194878 time to create 1 rle with new method : 0.008709192276000977 length of segment : 895 time for calcul the mask position with numpy : 0.0007405281066894531 nb_pixel_total : 7994 time to create 1 rle with old method : 0.0098114013671875 length of segment : 114 time for calcul the mask position with numpy : 0.0006856918334960938 nb_pixel_total : 14198 time to create 1 rle with old method : 0.017055749893188477 length of segment : 178 time for calcul the mask position with numpy : 0.0001697540283203125 nb_pixel_total : 6521 time to create 1 rle with old method : 0.011278629302978516 length of segment : 121 time for calcul the mask position with numpy : 0.0037310123443603516 nb_pixel_total : 115470 time to create 1 rle with old method : 0.13170266151428223 length of segment : 546 time for calcul the mask position with numpy : 0.005308389663696289 nb_pixel_total : 202518 time to create 1 rle with new method : 0.007107734680175781 length of segment : 778 time for calcul the mask position with numpy : 0.0005099773406982422 nb_pixel_total : 7540 time to create 1 rle with old method : 0.009515762329101562 length of segment : 121 time for calcul the mask position with numpy : 0.0036590099334716797 nb_pixel_total : 96773 time to create 1 rle with old method : 0.11066460609436035 length of segment : 524 time for calcul the mask position with numpy : 0.0003497600555419922 nb_pixel_total : 5464 time to create 1 rle with old method : 0.006716012954711914 length of segment : 141 time for calcul the mask position with numpy : 0.0010209083557128906 nb_pixel_total : 37701 time to create 1 rle with old method : 0.04383730888366699 length of segment : 234 time for calcul the mask position with numpy : 0.0003445148468017578 nb_pixel_total : 4263 time to create 1 rle with old method : 0.005327463150024414 length of segment : 131 time for calcul the mask position with numpy : 0.0009095668792724609 nb_pixel_total : 36835 time to create 1 rle with old method : 0.04338693618774414 length of segment : 188 time for calcul the mask position with numpy : 0.0015702247619628906 nb_pixel_total : 46281 time to create 1 rle with old method : 0.05261683464050293 length of segment : 265 time for calcul the mask position with numpy : 0.0003981590270996094 nb_pixel_total : 6448 time to create 1 rle with old method : 0.007931709289550781 length of segment : 113 time for calcul the mask position with numpy : 0.0003535747528076172 nb_pixel_total : 4207 time to create 1 rle with old method : 0.007298707962036133 length of segment : 104 time for calcul the mask position with numpy : 0.0006422996520996094 nb_pixel_total : 12222 time to create 1 rle with old method : 0.02033710479736328 length of segment : 172 time for calcul the mask position with numpy : 0.01567363739013672 nb_pixel_total : 517228 time to create 1 rle with new method : 0.024036645889282227 length of segment : 1111 time for calcul the mask position with numpy : 0.0007250308990478516 nb_pixel_total : 7858 time to create 1 rle with old method : 0.009895801544189453 length of segment : 241 time for calcul the mask position with numpy : 0.007049560546875 nb_pixel_total : 348569 time to create 1 rle with new method : 0.01253652572631836 length of segment : 568 time for calcul the mask position with numpy : 0.002251148223876953 nb_pixel_total : 79770 time to create 1 rle with old method : 0.09216022491455078 length of segment : 385 time for calcul the mask position with numpy : 0.002817869186401367 nb_pixel_total : 94939 time to create 1 rle with old method : 0.11106181144714355 length of segment : 372 time for calcul the mask position with numpy : 0.00042438507080078125 nb_pixel_total : 15398 time to create 1 rle with old method : 0.01810598373413086 length of segment : 220 time for calcul the mask position with numpy : 0.0012619495391845703 nb_pixel_total : 71598 time to create 1 rle with old method : 0.08303713798522949 length of segment : 361 time for calcul the mask position with numpy : 0.0006821155548095703 nb_pixel_total : 22796 time to create 1 rle with old method : 0.026726722717285156 length of segment : 267 time for calcul the mask position with numpy : 0.0019292831420898438 nb_pixel_total : 51008 time to create 1 rle with old method : 0.05903458595275879 length of segment : 462 time for calcul the mask position with numpy : 0.0008871555328369141 nb_pixel_total : 20649 time to create 1 rle with old method : 0.02388787269592285 length of segment : 273 time for calcul the mask position with numpy : 0.0005252361297607422 nb_pixel_total : 18385 time to create 1 rle with old method : 0.021515607833862305 length of segment : 158 time for calcul the mask position with numpy : 0.0029032230377197266 nb_pixel_total : 108069 time to create 1 rle with old method : 0.12294864654541016 length of segment : 571 time for calcul the mask position with numpy : 0.003405332565307617 nb_pixel_total : 151548 time to create 1 rle with new method : 0.0053250789642333984 length of segment : 401 time for calcul the mask position with numpy : 0.00022602081298828125 nb_pixel_total : 4638 time to create 1 rle with old method : 0.008255958557128906 length of segment : 56 time for calcul the mask position with numpy : 0.004302978515625 nb_pixel_total : 139446 time to create 1 rle with old method : 0.2009449005126953 length of segment : 490 time for calcul the mask position with numpy : 0.0030100345611572266 nb_pixel_total : 146035 time to create 1 rle with old method : 0.1880779266357422 length of segment : 502 time for calcul the mask position with numpy : 0.002444028854370117 nb_pixel_total : 58995 time to create 1 rle with old method : 0.07066845893859863 length of segment : 315 time for calcul the mask position with numpy : 0.0070188045501708984 nb_pixel_total : 261851 time to create 1 rle with new method : 0.011372804641723633 length of segment : 466 time for calcul the mask position with numpy : 0.00035858154296875 nb_pixel_total : 9190 time to create 1 rle with old method : 0.011055707931518555 length of segment : 116 time for calcul the mask position with numpy : 0.00010466575622558594 nb_pixel_total : 3602 time to create 1 rle with old method : 0.004446983337402344 length of segment : 72 time for calcul the mask position with numpy : 0.00037288665771484375 nb_pixel_total : 5907 time to create 1 rle with old method : 0.007489919662475586 length of segment : 75 time for calcul the mask position with numpy : 0.0007126331329345703 nb_pixel_total : 14021 time to create 1 rle with old method : 0.01622176170349121 length of segment : 258 time for calcul the mask position with numpy : 0.001413583755493164 nb_pixel_total : 55121 time to create 1 rle with old method : 0.06309318542480469 length of segment : 265 time for calcul the mask position with numpy : 0.001737356185913086 nb_pixel_total : 44102 time to create 1 rle with old method : 0.051111459732055664 length of segment : 257 time for calcul the mask position with numpy : 0.000453948974609375 nb_pixel_total : 14194 time to create 1 rle with old method : 0.023085594177246094 length of segment : 256 time for calcul the mask position with numpy : 0.0007781982421875 nb_pixel_total : 19813 time to create 1 rle with old method : 0.02382493019104004 length of segment : 203 time for calcul the mask position with numpy : 0.0037415027618408203 nb_pixel_total : 159812 time to create 1 rle with new method : 0.008884668350219727 length of segment : 382 time for calcul the mask position with numpy : 0.0001468658447265625 nb_pixel_total : 1437 time to create 1 rle with old method : 0.0019228458404541016 length of segment : 43 time for calcul the mask position with numpy : 0.0030159950256347656 nb_pixel_total : 124915 time to create 1 rle with old method : 0.14254498481750488 length of segment : 287 time for calcul the mask position with numpy : 0.0008444786071777344 nb_pixel_total : 41421 time to create 1 rle with old method : 0.04723858833312988 length of segment : 263 time for calcul the mask position with numpy : 0.00045752525329589844 nb_pixel_total : 7586 time to create 1 rle with old method : 0.009186744689941406 length of segment : 130 time for calcul the mask position with numpy : 0.0013155937194824219 nb_pixel_total : 33829 time to create 1 rle with old method : 0.04188942909240723 length of segment : 311 time for calcul the mask position with numpy : 0.06413459777832031 nb_pixel_total : 2606938 time to create 1 rle with new method : 0.12862086296081543 length of segment : 2962 time for calcul the mask position with numpy : 0.00022220611572265625 nb_pixel_total : 7973 time to create 1 rle with old method : 0.009789466857910156 length of segment : 219 time for calcul the mask position with numpy : 0.001340627670288086 nb_pixel_total : 77310 time to create 1 rle with old method : 0.08873486518859863 length of segment : 360 time for calcul the mask position with numpy : 0.000621795654296875 nb_pixel_total : 33988 time to create 1 rle with old method : 0.03951740264892578 length of segment : 213 time for calcul the mask position with numpy : 0.0001437664031982422 nb_pixel_total : 4070 time to create 1 rle with old method : 0.005018949508666992 length of segment : 67 time for calcul the mask position with numpy : 0.003306865692138672 nb_pixel_total : 144152 time to create 1 rle with old method : 0.16901564598083496 length of segment : 517 time for calcul the mask position with numpy : 0.003025054931640625 nb_pixel_total : 154494 time to create 1 rle with new method : 0.008979558944702148 length of segment : 609 time for calcul the mask position with numpy : 0.0025899410247802734 nb_pixel_total : 140066 time to create 1 rle with old method : 0.15665149688720703 length of segment : 818 time for calcul the mask position with numpy : 0.0008509159088134766 nb_pixel_total : 35191 time to create 1 rle with old method : 0.04005122184753418 length of segment : 193 time for calcul the mask position with numpy : 0.0032732486724853516 nb_pixel_total : 181427 time to create 1 rle with new method : 0.008724212646484375 length of segment : 929 time for calcul the mask position with numpy : 0.0004994869232177734 nb_pixel_total : 18168 time to create 1 rle with old method : 0.021175622940063477 length of segment : 153 time for calcul the mask position with numpy : 0.0018582344055175781 nb_pixel_total : 84580 time to create 1 rle with old method : 0.09637689590454102 length of segment : 662 time for calcul the mask position with numpy : 0.009003877639770508 nb_pixel_total : 423566 time to create 1 rle with new method : 0.024041414260864258 length of segment : 1131 time for calcul the mask position with numpy : 0.003774881362915039 nb_pixel_total : 126959 time to create 1 rle with old method : 0.15035128593444824 length of segment : 681 time for calcul the mask position with numpy : 0.0016584396362304688 nb_pixel_total : 55978 time to create 1 rle with old method : 0.06254744529724121 length of segment : 422 time for calcul the mask position with numpy : 0.009903907775878906 nb_pixel_total : 277623 time to create 1 rle with new method : 0.0192413330078125 length of segment : 539 time for calcul the mask position with numpy : 0.003116607666015625 nb_pixel_total : 102162 time to create 1 rle with old method : 0.11801528930664062 length of segment : 328 time for calcul the mask position with numpy : 0.03551077842712402 nb_pixel_total : 1109433 time to create 1 rle with new method : 0.07046008110046387 length of segment : 2309 time for calcul the mask position with numpy : 0.010664939880371094 nb_pixel_total : 567928 time to create 1 rle with new method : 0.021468400955200195 length of segment : 1078 time for calcul the mask position with numpy : 0.0010895729064941406 nb_pixel_total : 40668 time to create 1 rle with old method : 0.046013832092285156 length of segment : 286 time for calcul the mask position with numpy : 0.002808094024658203 nb_pixel_total : 138819 time to create 1 rle with old method : 0.1583406925201416 length of segment : 471 time for calcul the mask position with numpy : 0.002693653106689453 nb_pixel_total : 146438 time to create 1 rle with old method : 0.16457271575927734 length of segment : 396 time for calcul the mask position with numpy : 0.0001964569091796875 nb_pixel_total : 7373 time to create 1 rle with old method : 0.008849620819091797 length of segment : 136 time for calcul the mask position with numpy : 0.013551712036132812 nb_pixel_total : 683128 time to create 1 rle with new method : 0.03675985336303711 length of segment : 1240 time for calcul the mask position with numpy : 0.006722688674926758 nb_pixel_total : 269540 time to create 1 rle with new method : 0.8831081390380859 length of segment : 533 time for calcul the mask position with numpy : 0.003308534622192383 nb_pixel_total : 184949 time to create 1 rle with new method : 0.008599519729614258 length of segment : 494 time for calcul the mask position with numpy : 0.0016543865203857422 nb_pixel_total : 74569 time to create 1 rle with old method : 0.092041015625 length of segment : 245 time for calcul the mask position with numpy : 0.004488706588745117 nb_pixel_total : 149184 time to create 1 rle with old method : 0.17032933235168457 length of segment : 382 time for calcul the mask position with numpy : 0.0044596195220947266 nb_pixel_total : 138891 time to create 1 rle with old method : 0.15615344047546387 length of segment : 600 time for calcul the mask position with numpy : 0.002001523971557617 nb_pixel_total : 66244 time to create 1 rle with old method : 0.07597684860229492 length of segment : 324 time for calcul the mask position with numpy : 0.003449678421020508 nb_pixel_total : 138772 time to create 1 rle with old method : 0.15497469902038574 length of segment : 481 time for calcul the mask position with numpy : 0.0007545948028564453 nb_pixel_total : 24532 time to create 1 rle with old method : 0.02827620506286621 length of segment : 264 time for calcul the mask position with numpy : 0.00030040740966796875 nb_pixel_total : 3722 time to create 1 rle with old method : 0.004710197448730469 length of segment : 146 time for calcul the mask position with numpy : 0.0018467903137207031 nb_pixel_total : 118010 time to create 1 rle with old method : 0.13262391090393066 length of segment : 249 time for calcul the mask position with numpy : 0.0006437301635742188 nb_pixel_total : 8806 time to create 1 rle with old method : 0.011308670043945312 length of segment : 276 time for calcul the mask position with numpy : 0.0009124279022216797 nb_pixel_total : 44830 time to create 1 rle with old method : 0.05187058448791504 length of segment : 252 time for calcul the mask position with numpy : 0.002828359603881836 nb_pixel_total : 171830 time to create 1 rle with new method : 0.0050907135009765625 length of segment : 503 time for calcul the mask position with numpy : 0.014212369918823242 nb_pixel_total : 822210 time to create 1 rle with new method : 0.11329460144042969 length of segment : 1323 time for calcul the mask position with numpy : 0.000164031982421875 nb_pixel_total : 3550 time to create 1 rle with old method : 0.00442051887512207 length of segment : 42 time for calcul the mask position with numpy : 0.0013954639434814453 nb_pixel_total : 61667 time to create 1 rle with old method : 0.06969761848449707 length of segment : 437 time for calcul the mask position with numpy : 0.001130819320678711 nb_pixel_total : 67243 time to create 1 rle with old method : 0.07714176177978516 length of segment : 290 time for calcul the mask position with numpy : 0.09018874168395996 nb_pixel_total : 1532219 time to create 1 rle with new method : 0.18067121505737305 length of segment : 1033 time for calcul the mask position with numpy : 0.0008950233459472656 nb_pixel_total : 49310 time to create 1 rle with old method : 0.05968117713928223 length of segment : 311 time for calcul the mask position with numpy : 0.0021240711212158203 nb_pixel_total : 168774 time to create 1 rle with new method : 0.004506349563598633 length of segment : 465 time for calcul the mask position with numpy : 0.0001533031463623047 nb_pixel_total : 4574 time to create 1 rle with old method : 0.0056684017181396484 length of segment : 84 time for calcul the mask position with numpy : 0.13710975646972656 nb_pixel_total : 3207603 time to create 1 rle with new method : 0.2521982192993164 length of segment : 3460 time spent for convertir_results : 19.990933179855347 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 162 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 62132 save missing photos in datou_result : time spend for datou_step_exec : 68.23077321052551 time spend to save output : 6.919123649597168 total time spend for step 1 : 75.14989686012268 step2:crop_condition Tue Feb 11 03:29:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 162 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 14 About to insert : list_path_to_insert length 14 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 48 About to insert : list_path_to_insert length 48 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1336467271, 1336467269, 1336467265, 1336467262, 1336467258, 1336467218, 1336467216, 1336467168, 1336467164, 1336467153] Looping around the photos to save general results len do output : 97 /-3669082053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3669082052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102594Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102602Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102620Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102630Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102632Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102650Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124060Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102600Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102611Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102617Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102644Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102645Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102661Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3669082057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124082Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102590Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102597Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102612Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102603Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102589Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102599Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102586Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102593Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102592Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102618Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102606Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102615Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102608Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102626Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102629Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3669082054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102628Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102637Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102639Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102642Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102643Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102640Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102663Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124080Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3669082058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3669082059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102609Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102633Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102638Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102657Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102598Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102610Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102625Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102652Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102665Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668102664Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3668124038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467271', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467269', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467265', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467262', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467258', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467218', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467216', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467168', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467164', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467153', None, None, None, None, None, '2574456') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 301 time used for this insertion : 0.03809952735900879 save_final save missing photos in datou_result : time spend for datou_step_exec : 48.1817421913147 time spend to save output : 0.04041171073913574 total time spend for step 2 : 48.22215390205383 step3:thcl Tue Feb 11 03:30:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.014658451080322266 time to convert the images to numpy array : 0.10426807403564453 time to import caffe and check if the image exist : 0.01017451286315918 time to convert the images to numpy array : 0.2653944492340088 time to import caffe and check if the image exist : 0.014989614486694336 time to convert the images to numpy array : 0.266002893447876 time to import caffe and check if the image exist : 0.01232767105102539 time to convert the images to numpy array : 0.28531455993652344 time to import caffe and check if the image exist : 0.016265869140625 time to convert the images to numpy array : 0.2832217216491699 time to import caffe and check if the image exist : 0.0073397159576416016 time to convert the images to numpy array : 0.2968714237213135 time to import caffe and check if the image exist : 0.010265827178955078 time to convert the images to numpy array : 0.3040468692779541 time to import caffe and check if the image exist : 0.013618230819702148 time to convert the images to numpy array : 0.3116569519042969 time to import caffe and check if the image exist : 0.015455484390258789 time to convert the images to numpy array : 0.31443357467651367 time to import caffe and check if the image exist : 0.008418798446655273 time to convert the images to numpy array : 0.6712145805358887 total time to convert the images to numpy array : 0.8705034255981445 list photo_ids error: [] list photo_ids correct : [-3668124079, -3669082059, -3668124081, -3668102609, -3668102633, -3668102638, -3668102657, -3668124035, -3668102598, -3668102610, -3668102625, -3668102652, -3668102665, -3668124031, -3668124088, -3668102664, -3668124038, -3668124039, -3668124060, -3668102600, -3668102611, -3668102617, -3668102644, -3668102645, -3668102662, -3668102661, -3668102655, -3668102639, -3668102642, -3668102643, -3668102640, -3668102669, -3668102663, -3668102654, -3668102667, -3668124025, -3668124042, -3668102647, -3668124036, -3668124055, -3668124073, -3668124076, -3668102589, -3668102599, -3668102586, -3668102593, -3668102592, -3669082053, -3669082052, -3668102594, -3668102602, -3668102620, -3668102630, -3668102632, -3668102650, -3668124026, -3668124028, -3668124070, -3668124071, -3668124075, -3668124067, -3668124066, -3668124062, -3668124074, -3668124065, -3668124080, -3669082058, -3668124043, -3669082057, -3668124057, -3668124049, -3668124082, -3668124077, -3668102590, -3668102597, -3668102612, -3668102603, -3668102618, -3668102606, -3668102615, -3668102608, -3668102626, -3668102629, -3669082054, -3668102628, -3668102637, -3668102648, -3668124040, -3668124029, -3668124030, -3668124037, -3668124052, -3668124048, -3668124053, -3668124045, -3668124054, -3668124064] number of photos to traite : 97 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10553 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10334 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.2006475925445557 time used to do the prediction : 0.20903301239013672 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.274585485458374 time used to do the prediction : 0.17182326316833496 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.2172927856445312e-05 save missing photos in datou_result : time spend for datou_step_exec : 10.54116415977478 time spend to save output : 0.0008389949798583984 total time spend for step 3 : 10.542003154754639 step4:merge_mask_thcl_custom Tue Feb 11 03:30:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 162 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1336467271, 1336467269, 1336467265, 1336467262, 1336467258, 1336467218, 1336467216, 1336467168, 1336467164, 1336467153] Looping around the photos to save general results len do output : 10 /1336467271Didn't retrieve data .Didn't retrieve data . /1336467269Didn't retrieve data .Didn't retrieve data . /1336467265Didn't retrieve data .Didn't retrieve data . /1336467262Didn't retrieve data .Didn't retrieve data . /1336467258Didn't retrieve data .Didn't retrieve data . /1336467218Didn't retrieve data .Didn't retrieve data . /1336467216Didn't retrieve data .Didn't retrieve data . /1336467168Didn't retrieve data .Didn't retrieve data . /1336467164Didn't retrieve data .Didn't retrieve data . /1336467153Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467271', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467269', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467265', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467262', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467258', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467218', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467216', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467168', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467164', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467153', None, None, None, None, None, '2574456') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.301133394241333 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.1044621467590332 time spend to save output : 0.3015477657318115 total time spend for step 4 : 0.4060099124908447 step5:rle_unique_nms_with_priority Tue Feb 11 03:30:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 18 nb_hashtags : 6 time to prepare the origin masks : 12.896045207977295 time for calcul the mask position with numpy : 0.4995901584625244 nb_pixel_total : 6511385 time to create 1 rle with new method : 1.7321457862854004 time for calcul the mask position with numpy : 0.024050474166870117 nb_pixel_total : 30276 time to create 1 rle with old method : 0.035627126693725586 time for calcul the mask position with numpy : 0.02445697784423828 nb_pixel_total : 18049 time to create 1 rle with old method : 0.021806001663208008 time for calcul the mask position with numpy : 0.025043010711669922 nb_pixel_total : 6988 time to create 1 rle with old method : 0.008469343185424805 time for calcul the mask position with numpy : 0.024420976638793945 nb_pixel_total : 52381 time to create 1 rle with old method : 0.05954384803771973 time for calcul the mask position with numpy : 0.023625612258911133 nb_pixel_total : 27655 time to create 1 rle with old method : 0.034826040267944336 time for calcul the mask position with numpy : 0.022164583206176758 nb_pixel_total : 3061 time to create 1 rle with old method : 0.0037496089935302734 time for calcul the mask position with numpy : 0.02301621437072754 nb_pixel_total : 78924 time to create 1 rle with old method : 0.08890676498413086 time for calcul the mask position with numpy : 0.022669076919555664 nb_pixel_total : 4448 time to create 1 rle with old method : 0.005175113677978516 time for calcul the mask position with numpy : 0.022854089736938477 nb_pixel_total : 2459 time to create 1 rle with old method : 0.0029754638671875 time for calcul the mask position with numpy : 0.03385019302368164 nb_pixel_total : 10573 time to create 1 rle with old method : 0.012218236923217773 time for calcul the mask position with numpy : 0.038701772689819336 nb_pixel_total : 16113 time to create 1 rle with old method : 0.020271778106689453 time for calcul the mask position with numpy : 0.037545204162597656 nb_pixel_total : 37 time to create 1 rle with old method : 8.368492126464844e-05 time for calcul the mask position with numpy : 0.038321495056152344 nb_pixel_total : 161682 time to create 1 rle with new method : 0.8608105182647705 time for calcul the mask position with numpy : 0.0371546745300293 nb_pixel_total : 48980 time to create 1 rle with old method : 0.05566263198852539 time for calcul the mask position with numpy : 0.034429311752319336 nb_pixel_total : 55082 time to create 1 rle with old method : 0.06994199752807617 time for calcul the mask position with numpy : 0.03108811378479004 nb_pixel_total : 3354 time to create 1 rle with old method : 0.004056453704833984 time for calcul the mask position with numpy : 0.025234460830688477 nb_pixel_total : 6498 time to create 1 rle with old method : 0.007567167282104492 time for calcul the mask position with numpy : 0.023161649703979492 nb_pixel_total : 12295 time to create 1 rle with old method : 0.014040708541870117 create new chi : 4.115107536315918 time to delete rle : 0.01069188117980957 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 10633 TO DO : save crop sub photo not yet done ! save time : 0.97515869140625 nb_obj : 15 nb_hashtags : 4 time to prepare the origin masks : 10.09275221824646 time for calcul the mask position with numpy : 0.5039739608764648 nb_pixel_total : 6059190 time to create 1 rle with new method : 1.1922554969787598 time for calcul the mask position with numpy : 0.04214048385620117 nb_pixel_total : 19148 time to create 1 rle with old method : 0.02119922637939453 time for calcul the mask position with numpy : 0.04008626937866211 nb_pixel_total : 677 time to create 1 rle with old method : 0.0009579658508300781 time for calcul the mask position with numpy : 0.04758787155151367 nb_pixel_total : 190350 time to create 1 rle with new method : 0.8688709735870361 time for calcul the mask position with numpy : 0.051337480545043945 nb_pixel_total : 84889 time to create 1 rle with old method : 0.0963444709777832 time for calcul the mask position with numpy : 0.04954028129577637 nb_pixel_total : 39467 time to create 1 rle with old method : 0.04478192329406738 time for calcul the mask position with numpy : 0.04492521286010742 nb_pixel_total : 3277 time to create 1 rle with old method : 0.0043599605560302734 time for calcul the mask position with numpy : 0.042119503021240234 nb_pixel_total : 32712 time to create 1 rle with old method : 0.03795146942138672 time for calcul the mask position with numpy : 0.04339718818664551 nb_pixel_total : 38286 time to create 1 rle with old method : 0.046672821044921875 time for calcul the mask position with numpy : 0.0384516716003418 nb_pixel_total : 842 time to create 1 rle with old method : 0.0009958744049072266 time for calcul the mask position with numpy : 0.03731679916381836 nb_pixel_total : 49564 time to create 1 rle with old method : 0.05486035346984863 time for calcul the mask position with numpy : 0.04513359069824219 nb_pixel_total : 19274 time to create 1 rle with old method : 0.02288365364074707 time for calcul the mask position with numpy : 0.04369497299194336 nb_pixel_total : 176413 time to create 1 rle with new method : 0.6824274063110352 time for calcul the mask position with numpy : 0.050640106201171875 nb_pixel_total : 144386 time to create 1 rle with old method : 0.1662437915802002 time for calcul the mask position with numpy : 0.04189729690551758 nb_pixel_total : 66727 time to create 1 rle with old method : 0.08208012580871582 time for calcul the mask position with numpy : 0.03620290756225586 nb_pixel_total : 125038 time to create 1 rle with old method : 0.1405177116394043 create new chi : 4.74200439453125 time to delete rle : 0.001953601837158203 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 13608 TO DO : save crop sub photo not yet done ! save time : 1.4616830348968506 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 10.808362007141113 time for calcul the mask position with numpy : 0.5292127132415771 nb_pixel_total : 6499768 time to create 1 rle with new method : 0.6018271446228027 time for calcul the mask position with numpy : 0.038008928298950195 nb_pixel_total : 100208 time to create 1 rle with old method : 0.11458325386047363 time for calcul the mask position with numpy : 0.041826486587524414 nb_pixel_total : 47 time to create 1 rle with old method : 0.00016307830810546875 time for calcul the mask position with numpy : 0.04171013832092285 nb_pixel_total : 4514 time to create 1 rle with old method : 0.007297515869140625 time for calcul the mask position with numpy : 0.04454851150512695 nb_pixel_total : 91060 time to create 1 rle with old method : 0.11540007591247559 time for calcul the mask position with numpy : 0.03772735595703125 nb_pixel_total : 111366 time to create 1 rle with old method : 0.12663650512695312 time for calcul the mask position with numpy : 0.038941383361816406 nb_pixel_total : 30102 time to create 1 rle with old method : 0.03384065628051758 time for calcul the mask position with numpy : 0.03639030456542969 nb_pixel_total : 4949 time to create 1 rle with old method : 0.006383180618286133 time for calcul the mask position with numpy : 0.04125189781188965 nb_pixel_total : 12100 time to create 1 rle with old method : 0.014604568481445312 time for calcul the mask position with numpy : 0.03594207763671875 nb_pixel_total : 155137 time to create 1 rle with new method : 0.7834773063659668 time for calcul the mask position with numpy : 0.04165244102478027 nb_pixel_total : 21239 time to create 1 rle with old method : 0.02441883087158203 time for calcul the mask position with numpy : 0.039697885513305664 nb_pixel_total : 9796 time to create 1 rle with old method : 0.013512849807739258 time for calcul the mask position with numpy : 0.0456385612487793 nb_pixel_total : 9954 time to create 1 rle with old method : 0.011467933654785156 create new chi : 2.951289415359497 time to delete rle : 0.0014345645904541016 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 9330 TO DO : save crop sub photo not yet done ! save time : 1.1586236953735352 nb_obj : 18 nb_hashtags : 6 time to prepare the origin masks : 12.643950700759888 time for calcul the mask position with numpy : 1.4738473892211914 nb_pixel_total : 6025569 time to create 1 rle with new method : 0.7454123497009277 time for calcul the mask position with numpy : 0.02986884117126465 nb_pixel_total : 14123 time to create 1 rle with old method : 0.01536703109741211 time for calcul the mask position with numpy : 0.0312347412109375 nb_pixel_total : 18874 time to create 1 rle with old method : 0.02077341079711914 time for calcul the mask position with numpy : 0.0321652889251709 nb_pixel_total : 38645 time to create 1 rle with old method : 0.04106307029724121 time for calcul the mask position with numpy : 0.03734302520751953 nb_pixel_total : 24694 time to create 1 rle with old method : 0.029882192611694336 time for calcul the mask position with numpy : 0.030783414840698242 nb_pixel_total : 78160 time to create 1 rle with old method : 0.08602619171142578 time for calcul the mask position with numpy : 0.04099845886230469 nb_pixel_total : 13589 time to create 1 rle with old method : 0.0168914794921875 time for calcul the mask position with numpy : 0.03262925148010254 nb_pixel_total : 12010 time to create 1 rle with old method : 0.012735128402709961 time for calcul the mask position with numpy : 0.0334925651550293 nb_pixel_total : 93202 time to create 1 rle with old method : 0.0987098217010498 time for calcul the mask position with numpy : 0.032746315002441406 nb_pixel_total : 99201 time to create 1 rle with old method : 0.10939598083496094 time for calcul the mask position with numpy : 0.039502620697021484 nb_pixel_total : 3309 time to create 1 rle with old method : 0.00406956672668457 time for calcul the mask position with numpy : 0.03275179862976074 nb_pixel_total : 126716 time to create 1 rle with old method : 0.1398167610168457 time for calcul the mask position with numpy : 0.032942771911621094 nb_pixel_total : 202026 time to create 1 rle with new method : 0.42873191833496094 time for calcul the mask position with numpy : 0.03947329521179199 nb_pixel_total : 1163 time to create 1 rle with old method : 0.0014331340789794922 time for calcul the mask position with numpy : 0.02395319938659668 nb_pixel_total : 4613 time to create 1 rle with old method : 0.005438089370727539 time for calcul the mask position with numpy : 0.024694442749023438 nb_pixel_total : 207907 time to create 1 rle with new method : 0.38628649711608887 time for calcul the mask position with numpy : 0.022160768508911133 nb_pixel_total : 7946 time to create 1 rle with old method : 0.009007453918457031 time for calcul the mask position with numpy : 0.022582292556762695 nb_pixel_total : 72793 time to create 1 rle with old method : 0.0784292221069336 time for calcul the mask position with numpy : 0.02169060707092285 nb_pixel_total : 5700 time to create 1 rle with old method : 0.006776332855224609 create new chi : 4.359222173690796 time to delete rle : 0.002363443374633789 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 12755 TO DO : save crop sub photo not yet done ! save time : 3.388967514038086 nb_obj : 19 nb_hashtags : 6 time to prepare the origin masks : 11.004479169845581 time for calcul the mask position with numpy : 0.3555471897125244 nb_pixel_total : 5657906 time to create 1 rle with new method : 0.6625218391418457 time for calcul the mask position with numpy : 0.04193902015686035 nb_pixel_total : 4173 time to create 1 rle with old method : 0.005848884582519531 time for calcul the mask position with numpy : 0.03581571578979492 nb_pixel_total : 36780 time to create 1 rle with old method : 0.04177260398864746 time for calcul the mask position with numpy : 0.03878307342529297 nb_pixel_total : 52638 time to create 1 rle with old method : 0.06853246688842773 time for calcul the mask position with numpy : 0.037026405334472656 nb_pixel_total : 18267 time to create 1 rle with old method : 0.02062678337097168 time for calcul the mask position with numpy : 0.0378265380859375 nb_pixel_total : 20504 time to create 1 rle with old method : 0.02358412742614746 time for calcul the mask position with numpy : 0.04286074638366699 nb_pixel_total : 37571 time to create 1 rle with old method : 0.042455434799194336 time for calcul the mask position with numpy : 0.036595821380615234 nb_pixel_total : 3962 time to create 1 rle with old method : 0.004918336868286133 time for calcul the mask position with numpy : 0.04319596290588379 nb_pixel_total : 516771 time to create 1 rle with new method : 0.5253968238830566 time for calcul the mask position with numpy : 0.03704428672790527 nb_pixel_total : 46224 time to create 1 rle with old method : 0.05180168151855469 time for calcul the mask position with numpy : 0.03257608413696289 nb_pixel_total : 11788 time to create 1 rle with old method : 0.016095399856567383 time for calcul the mask position with numpy : 0.037683725357055664 nb_pixel_total : 6409 time to create 1 rle with old method : 0.007582426071166992 time for calcul the mask position with numpy : 0.034725189208984375 nb_pixel_total : 347777 time to create 1 rle with new method : 0.6645364761352539 time for calcul the mask position with numpy : 0.030646800994873047 nb_pixel_total : 96709 time to create 1 rle with old method : 0.10956668853759766 time for calcul the mask position with numpy : 0.029402494430541992 nb_pixel_total : 4250 time to create 1 rle with old method : 0.005200862884521484 time for calcul the mask position with numpy : 0.036736249923706055 nb_pixel_total : 7310 time to create 1 rle with old method : 0.01254892349243164 time for calcul the mask position with numpy : 0.03586554527282715 nb_pixel_total : 94646 time to create 1 rle with old method : 0.11142969131469727 time for calcul the mask position with numpy : 0.03666281700134277 nb_pixel_total : 9655 time to create 1 rle with old method : 0.012206554412841797 time for calcul the mask position with numpy : 0.039936065673828125 nb_pixel_total : 5434 time to create 1 rle with old method : 0.00630640983581543 time for calcul the mask position with numpy : 0.037569284439086914 nb_pixel_total : 71466 time to create 1 rle with old method : 0.0830073356628418 create new chi : 3.6350061893463135 time to delete rle : 0.0020971298217773438 batch 1 Loaded 21 chid ids of type : 3726 Number RLEs to save : 13471 TO DO : save crop sub photo not yet done ! save time : 1.6496450901031494 nb_obj : 19 nb_hashtags : 7 time to prepare the origin masks : 10.239595651626587 time for calcul the mask position with numpy : 0.499037504196167 nb_pixel_total : 6032996 time to create 1 rle with new method : 0.5015735626220703 time for calcul the mask position with numpy : 0.03545713424682617 nb_pixel_total : 58924 time to create 1 rle with old method : 0.06631112098693848 time for calcul the mask position with numpy : 0.03563332557678223 nb_pixel_total : 106898 time to create 1 rle with old method : 0.11970162391662598 time for calcul the mask position with numpy : 0.03471851348876953 nb_pixel_total : 54977 time to create 1 rle with old method : 0.05913734436035156 time for calcul the mask position with numpy : 0.03519582748413086 nb_pixel_total : 19748 time to create 1 rle with old method : 0.022819042205810547 time for calcul the mask position with numpy : 0.04091477394104004 nb_pixel_total : 261558 time to create 1 rle with new method : 0.5390810966491699 time for calcul the mask position with numpy : 0.03852081298828125 nb_pixel_total : 6897 time to create 1 rle with old method : 0.008591890335083008 time for calcul the mask position with numpy : 0.03525733947753906 nb_pixel_total : 1425 time to create 1 rle with old method : 0.0017325878143310547 time for calcul the mask position with numpy : 0.03526639938354492 nb_pixel_total : 1414 time to create 1 rle with old method : 0.0020635128021240234 time for calcul the mask position with numpy : 0.03705739974975586 nb_pixel_total : 159629 time to create 1 rle with new method : 1.261061668395996 time for calcul the mask position with numpy : 0.03565621376037598 nb_pixel_total : 139265 time to create 1 rle with old method : 0.1651313304901123 time for calcul the mask position with numpy : 0.03997611999511719 nb_pixel_total : 643 time to create 1 rle with old method : 0.0010442733764648438 time for calcul the mask position with numpy : 0.040227651596069336 nb_pixel_total : 5872 time to create 1 rle with old method : 0.009502410888671875 time for calcul the mask position with numpy : 0.04185175895690918 nb_pixel_total : 3591 time to create 1 rle with old method : 0.00591278076171875 time for calcul the mask position with numpy : 0.038103580474853516 nb_pixel_total : 9152 time to create 1 rle with old method : 0.01138758659362793 time for calcul the mask position with numpy : 0.03607630729675293 nb_pixel_total : 43989 time to create 1 rle with old method : 0.049257755279541016 time for calcul the mask position with numpy : 0.03417396545410156 nb_pixel_total : 303 time to create 1 rle with old method : 0.0005118846893310547 time for calcul the mask position with numpy : 0.03391385078430176 nb_pixel_total : 14015 time to create 1 rle with old method : 0.01593327522277832 time for calcul the mask position with numpy : 0.03309440612792969 nb_pixel_total : 4597 time to create 1 rle with old method : 0.0055446624755859375 time for calcul the mask position with numpy : 0.026337146759033203 nb_pixel_total : 124347 time to create 1 rle with old method : 0.14279818534851074 create new chi : 4.268608570098877 time to delete rle : 0.0015513896942138672 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 11108 TO DO : save crop sub photo not yet done ! save time : 1.6409788131713867 nb_obj : 16 nb_hashtags : 4 time to prepare the origin masks : 12.245855331420898 time for calcul the mask position with numpy : 0.5245723724365234 nb_pixel_total : 3584547 time to create 1 rle with new method : 0.783710241317749 time for calcul the mask position with numpy : 0.043320417404174805 nb_pixel_total : 242093 time to create 1 rle with new method : 0.46392369270324707 time for calcul the mask position with numpy : 0.041345834732055664 nb_pixel_total : 1407 time to create 1 rle with old method : 0.0021147727966308594 time for calcul the mask position with numpy : 0.04106497764587402 nb_pixel_total : 87086 time to create 1 rle with old method : 0.1041867733001709 time for calcul the mask position with numpy : 0.03702259063720703 nb_pixel_total : 33707 time to create 1 rle with old method : 0.03763866424560547 time for calcul the mask position with numpy : 0.0424039363861084 nb_pixel_total : 18129 time to create 1 rle with old method : 0.020787954330444336 time for calcul the mask position with numpy : 0.04549694061279297 nb_pixel_total : 35048 time to create 1 rle with old method : 0.041742563247680664 time for calcul the mask position with numpy : 0.04056859016418457 nb_pixel_total : 77135 time to create 1 rle with old method : 0.0899045467376709 time for calcul the mask position with numpy : 0.03472256660461426 nb_pixel_total : 4050 time to create 1 rle with old method : 0.004730701446533203 time for calcul the mask position with numpy : 0.03313112258911133 nb_pixel_total : 1236 time to create 1 rle with old method : 0.001543283462524414 time for calcul the mask position with numpy : 0.03362298011779785 nb_pixel_total : 116884 time to create 1 rle with old method : 0.1306915283203125 time for calcul the mask position with numpy : 0.038568735122680664 nb_pixel_total : 83096 time to create 1 rle with old method : 0.09126830101013184 time for calcul the mask position with numpy : 0.03366732597351074 nb_pixel_total : 111309 time to create 1 rle with old method : 0.1316695213317871 time for calcul the mask position with numpy : 0.03564000129699707 nb_pixel_total : 33676 time to create 1 rle with old method : 0.037268877029418945 time for calcul the mask position with numpy : 0.032167911529541016 nb_pixel_total : 7968 time to create 1 rle with old method : 0.008923768997192383 time for calcul the mask position with numpy : 0.058492183685302734 nb_pixel_total : 2605334 time to create 1 rle with new method : 0.7063314914703369 time for calcul the mask position with numpy : 0.03821873664855957 nb_pixel_total : 7535 time to create 1 rle with old method : 0.009912967681884766 create new chi : 3.921884298324585 time to delete rle : 0.0025336742401123047 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 21906 TO DO : save crop sub photo not yet done ! save time : 2.7057857513427734 nb_obj : 18 nb_hashtags : 6 time to prepare the origin masks : 13.028950452804565 time for calcul the mask position with numpy : 0.20198297500610352 nb_pixel_total : 4247240 time to create 1 rle with new method : 0.5380141735076904 time for calcul the mask position with numpy : 0.050330162048339844 nb_pixel_total : 66081 time to create 1 rle with old method : 0.07516694068908691 time for calcul the mask position with numpy : 0.04038882255554199 nb_pixel_total : 3654 time to create 1 rle with old method : 0.004873752593994141 time for calcul the mask position with numpy : 0.042955636978149414 nb_pixel_total : 146136 time to create 1 rle with old method : 0.17049765586853027 time for calcul the mask position with numpy : 0.05043601989746094 nb_pixel_total : 74411 time to create 1 rle with old method : 0.08638548851013184 time for calcul the mask position with numpy : 0.03180360794067383 nb_pixel_total : 204221 time to create 1 rle with new method : 0.9750754833221436 time for calcul the mask position with numpy : 0.03991198539733887 nb_pixel_total : 18581 time to create 1 rle with old method : 0.02182459831237793 time for calcul the mask position with numpy : 0.042081594467163086 nb_pixel_total : 138705 time to create 1 rle with old method : 0.15740680694580078 time for calcul the mask position with numpy : 0.034844398498535156 nb_pixel_total : 7357 time to create 1 rle with old method : 0.00826573371887207 time for calcul the mask position with numpy : 0.03608083724975586 nb_pixel_total : 237680 time to create 1 rle with new method : 0.46965551376342773 time for calcul the mask position with numpy : 0.040277719497680664 nb_pixel_total : 40602 time to create 1 rle with old method : 0.045171499252319336 time for calcul the mask position with numpy : 0.0476231575012207 nb_pixel_total : 978307 time to create 1 rle with new method : 0.7331795692443848 time for calcul the mask position with numpy : 0.029553651809692383 nb_pixel_total : 2564 time to create 1 rle with old method : 0.003237009048461914 time for calcul the mask position with numpy : 0.03231668472290039 nb_pixel_total : 55972 time to create 1 rle with old method : 0.060338735580444336 time for calcul the mask position with numpy : 0.03174400329589844 nb_pixel_total : 101467 time to create 1 rle with old method : 0.1087639331817627 time for calcul the mask position with numpy : 0.029153108596801758 nb_pixel_total : 277101 time to create 1 rle with new method : 0.4008619785308838 time for calcul the mask position with numpy : 0.0350034236907959 nb_pixel_total : 184653 time to create 1 rle with new method : 1.1097981929779053 time for calcul the mask position with numpy : 0.03353261947631836 nb_pixel_total : 126731 time to create 1 rle with old method : 0.1443018913269043 time for calcul the mask position with numpy : 0.035239458084106445 nb_pixel_total : 138777 time to create 1 rle with old method : 0.1532421112060547 create new chi : 6.330825567245483 time to delete rle : 0.0025854110717773438 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 19984 TO DO : save crop sub photo not yet done ! save time : 2.945409059524536 nb_obj : 13 nb_hashtags : 7 time to prepare the origin masks : 12.768166303634644 time for calcul the mask position with numpy : 0.48712706565856934 nb_pixel_total : 4188987 time to create 1 rle with new method : 1.5443825721740723 time for calcul the mask position with numpy : 0.029695510864257812 nb_pixel_total : 8707 time to create 1 rle with old method : 0.010442733764648438 time for calcul the mask position with numpy : 0.03506827354431152 nb_pixel_total : 3301 time to create 1 rle with old method : 0.004336357116699219 time for calcul the mask position with numpy : 0.054346561431884766 nb_pixel_total : 1531145 time to create 1 rle with new method : 1.092695951461792 time for calcul the mask position with numpy : 0.03465008735656738 nb_pixel_total : 117829 time to create 1 rle with old method : 0.13213181495666504 time for calcul the mask position with numpy : 0.03277325630187988 nb_pixel_total : 3518 time to create 1 rle with old method : 0.004059314727783203 time for calcul the mask position with numpy : 0.04425954818725586 nb_pixel_total : 820630 time to create 1 rle with new method : 1.292722463607788 time for calcul the mask position with numpy : 0.0343928337097168 nb_pixel_total : 61646 time to create 1 rle with old method : 0.06859493255615234 time for calcul the mask position with numpy : 0.026261568069458008 nb_pixel_total : 25411 time to create 1 rle with old method : 0.0290985107421875 time for calcul the mask position with numpy : 0.025583505630493164 nb_pixel_total : 3695 time to create 1 rle with old method : 0.005791187286376953 time for calcul the mask position with numpy : 0.04193925857543945 nb_pixel_total : 44697 time to create 1 rle with old method : 0.08173584938049316 time for calcul the mask position with numpy : 0.043996334075927734 nb_pixel_total : 4922 time to create 1 rle with old method : 0.009542226791381836 time for calcul the mask position with numpy : 0.04970574378967285 nb_pixel_total : 168630 time to create 1 rle with new method : 0.6713025569915771 time for calcul the mask position with numpy : 0.05294942855834961 nb_pixel_total : 67122 time to create 1 rle with old method : 0.11449885368347168 create new chi : 6.169808387756348 time to delete rle : 0.0012617111206054688 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 12258 TO DO : save crop sub photo not yet done ! save time : 1.6965088844299316 nb_obj : 2 nb_hashtags : 1 time to prepare the origin masks : 2.530360460281372 time for calcul the mask position with numpy : 0.5186114311218262 nb_pixel_total : 3846657 time to create 1 rle with new method : 0.559175968170166 time for calcul the mask position with numpy : 0.21981239318847656 nb_pixel_total : 3199045 time to create 1 rle with new method : 0.6418552398681641 time for calcul the mask position with numpy : 0.019238710403442383 nb_pixel_total : 4538 time to create 1 rle with old method : 0.0049896240234375 create new chi : 2.025477409362793 time to delete rle : 0.0008537769317626953 batch 1 Loaded 3 chid ids of type : 3726 Number RLEs to save : 9260 TO DO : save crop sub photo not yet done ! save time : 1.6762914657592773 map_output_result : {1336467271: (0.0, 'Should be the crop_list due to order', 0.0), 1336467269: (0.0, 'Should be the crop_list due to order', 0.0), 1336467265: (0.0, 'Should be the crop_list due to order', 0.0), 1336467262: (0.0, 'Should be the crop_list due to order', 0.0), 1336467258: (0.0, 'Should be the crop_list due to order', 0.0), 1336467218: (0.0, 'Should be the crop_list due to order', 0.0), 1336467216: (0.0, 'Should be the crop_list due to order', 0.0), 1336467168: (0.0, 'Should be the crop_list due to order', 0.0), 1336467164: (0.0, 'Should be the crop_list due to order', 0.0), 1336467153: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1336467271, 1336467269, 1336467265, 1336467262, 1336467258, 1336467218, 1336467216, 1336467168, 1336467164, 1336467153] Looping around the photos to save general results len do output : 10 /1336467271.Didn't retrieve data . /1336467269.Didn't retrieve data . /1336467265.Didn't retrieve data . /1336467262.Didn't retrieve data . /1336467258.Didn't retrieve data . /1336467218.Didn't retrieve data . /1336467216.Didn't retrieve data . /1336467168.Didn't retrieve data . /1336467164.Didn't retrieve data . /1336467153.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467271', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467269', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467265', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467262', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467258', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467218', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467216', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467168', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467164', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467153', None, None, None, None, None, '2574456') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.022372722625732422 save_final save missing photos in datou_result : time spend for datou_step_exec : 171.17164587974548 time spend to save output : 0.022876977920532227 total time spend for step 5 : 171.19452285766602 step6:crop_condition Tue Feb 11 03:33:34 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 170 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739241218_855890 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.8504633903503418 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739241226_855890 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.417508602142334 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739241232_855890 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.6316351890563965 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739241236_855890 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.5952057838439941 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739241241_855890 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.301337718963623 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739241244_855890 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.8111269474029541 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1336467271, 1336467269, 1336467265, 1336467262, 1336467258, 1336467218, 1336467216, 1336467168, 1336467164, 1336467153] Looping around the photos to save general results len do output : 35 /1336680914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680926Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680927Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680928Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680935Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680958Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680960Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680961Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680962Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680966Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680967Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680970Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336680972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467271', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467269', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467265', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467262', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467258', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467218', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467216', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467168', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467164', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467153', None, None, None, None, None, '2574456') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 115 time used for this insertion : 0.04309678077697754 save_final save missing photos in datou_result : time spend for datou_step_exec : 33.01225447654724 time spend to save output : 0.04418230056762695 total time spend for step 6 : 33.05643677711487 step7:ventilate_hashtags_in_portfolio Tue Feb 11 03:34:07 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20425795 get user id for portfolio 20425795 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20425795 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','autre_refus','plastique','flou','Teint_Dans_La_Masse','Carton_brun','environnement','metal','papier','mal_croppe','kraft','cartonnette')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20425795 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','autre_refus','plastique','flou','Teint_Dans_La_Masse','Carton_brun','environnement','metal','papier','mal_croppe','kraft','cartonnette')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20425913,20425914,20425915,20425916,20425917,20425918,20425919,20425920,20425921,20425922,20425923,20425924?tags=mal_croppe,autre_refus,Carton_brun,papier,flou,Teint_Dans_La_Masse,Carton_gris,cartonnette,kraft,metal,plastique,environnement Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1336467271, 1336467269, 1336467265, 1336467262, 1336467258, 1336467218, 1336467216, 1336467168, 1336467164, 1336467153] Looping around the photos to save general results len do output : 1 /20425795. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467271', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467269', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467265', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467262', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467258', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467218', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467216', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467168', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467164', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467153', None, None, None, None, None, '2574456') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.09220337867736816 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.683924674987793 time spend to save output : 0.09269332885742188 total time spend for step 7 : 1.7766180038452148 step8:final Tue Feb 11 03:34:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1336467271: ('0.08890214793955738',), 1336467269: ('0.08890214793955738',), 1336467265: ('0.08890214793955738',), 1336467262: ('0.08890214793955738',), 1336467258: ('0.08890214793955738',), 1336467218: ('0.08890214793955738',), 1336467216: ('0.08890214793955738',), 1336467168: ('0.08890214793955738',), 1336467164: ('0.08890214793955738',), 1336467153: ('0.08890214793955738',)} new output for save of step final : {1336467271: ('0.08890214793955738',), 1336467269: ('0.08890214793955738',), 1336467265: ('0.08890214793955738',), 1336467262: ('0.08890214793955738',), 1336467258: ('0.08890214793955738',), 1336467218: ('0.08890214793955738',), 1336467216: ('0.08890214793955738',), 1336467168: ('0.08890214793955738',), 1336467164: ('0.08890214793955738',), 1336467153: ('0.08890214793955738',)} [1336467271, 1336467269, 1336467265, 1336467262, 1336467258, 1336467218, 1336467216, 1336467168, 1336467164, 1336467153] Looping around the photos to save general results len do output : 10 /1336467271.Didn't retrieve data . /1336467269.Didn't retrieve data . /1336467265.Didn't retrieve data . /1336467262.Didn't retrieve data . /1336467258.Didn't retrieve data . /1336467218.Didn't retrieve data . /1336467216.Didn't retrieve data . /1336467168.Didn't retrieve data . /1336467164.Didn't retrieve data . /1336467153.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467271', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467269', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467265', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467262', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467258', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467218', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467216', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467168', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467164', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467153', None, None, None, None, None, '2574456') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.015795469284057617 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11142539978027344 time spend to save output : 0.01621699333190918 total time spend for step 8 : 0.12764239311218262 step9:velours_tree Tue Feb 11 03:34:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.1199500560760498 time spend to save output : 6.651878356933594e-05 total time spend for step 9 : 0.12001657485961914 step10:send_mail_cod Tue Feb 11 03:34:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20425795_11-02-2025_03_34_09.pdf 20425913 imagette204259131739241249 20425914 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204259141739241249 20425917 imagette204259171739241250 20425920 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204259201739241250 20425921 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204259211739241251 20425922 change filename to text .imagette204259221739241251 20425923 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204259231739241252 20425915 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204259151739241253 20425918 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204259181739241254 20425919 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette204259191739241256 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20425795 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20425913,20425914,20425915,20425916,20425917,20425918,20425919,20425920,20425921,20425922,20425923,20425924?tags=mal_croppe,autre_refus,Carton_brun,papier,flou,Teint_Dans_La_Masse,Carton_gris,cartonnette,kraft,metal,plastique,environnement your option no_mail is active, we will not send the real mail to your client args[1336467271] : ('0.08890214793955738',) no score found for photo 1336467271 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467269] : ('0.08890214793955738',) no score found for photo 1336467269 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467265] : ('0.08890214793955738',) no score found for photo 1336467265 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467262] : ('0.08890214793955738',) no score found for photo 1336467262 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467258] : ('0.08890214793955738',) no score found for photo 1336467258 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467218] : ('0.08890214793955738',) no score found for photo 1336467218 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467216] : ('0.08890214793955738',) no score found for photo 1336467216 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467168] : ('0.08890214793955738',) no score found for photo 1336467168 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467164] : ('0.08890214793955738',) no score found for photo 1336467164 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com args[1336467153] : ('0.08890214793955738',) no score found for photo 1336467153 apple ('0.08890214793955738',) We are sending mail with results at report@fotonower.com refus_total : 0.08890214793955738 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20425795 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1336467265,1336671109,1336671107,1336671105,1336671103,1336670850,1336468186,1336467272,1336467271,1336467269,1336467060,1336467262,1336467258,1336467218,1336467168,1336467128,1336467063,1336467216,1336467164,1336467313) Found this number of photos: 20 begin to download photo : 1336467265 begin to download photo : 1336670850 begin to download photo : 1336467060 begin to download photo : 1336467128 download finish for photo 1336670850 begin to download photo : 1336468186 download finish for photo 1336467060 begin to download photo : 1336467262 download finish for photo 1336467265 begin to download photo : 1336671109 download finish for photo 1336467128 begin to download photo : 1336467063 download finish for photo 1336468186 begin to download photo : 1336467272 download finish for photo 1336671109 begin to download photo : 1336671107 download finish for photo 1336467262 begin to download photo : 1336467258 download finish for photo 1336467272 begin to download photo : 1336467271 download finish for photo 1336467063 begin to download photo : 1336467216 download finish for photo 1336671107 begin to download photo : 1336671105 download finish for photo 1336467258 begin to download photo : 1336467218 download finish for photo 1336467216 begin to download photo : 1336467164 download finish for photo 1336467271 begin to download photo : 1336467269 download finish for photo 1336671105 begin to download photo : 1336671103 download finish for photo 1336467218 begin to download photo : 1336467168 download finish for photo 1336467269 download finish for photo 1336467164 begin to download photo : 1336467313 download finish for photo 1336671103 download finish for photo 1336467168 download finish for photo 1336467313 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425795_11-02-2025_03_34_09.pdf results_Auto_P20425795_11-02-2025_03_34_09.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425795_11-02-2025_03_34_09.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20425795','results_Auto_P20425795_11-02-2025_03_34_09.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425795_11-02-2025_03_34_09.pdf','pdf','','1.49','0.08890214793955738') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1336467271, 1336467269, 1336467265, 1336467262, 1336467258, 1336467218, 1336467216, 1336467168, 1336467164, 1336467153] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467271', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467269', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467265', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467262', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467258', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467218', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467216', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467168', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467164', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467153', None, None, None, None, None, '2574456') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.015195846557617188 save_final save missing photos in datou_result : time spend for datou_step_exec : 13.419233560562134 time spend to save output : 0.015480518341064453 total time spend for step 10 : 13.434714078903198 step11:split_time_score Tue Feb 11 03:34:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('11', 26),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10022025 20425795 Nombre de photos uploadées : 26 / 23040 (0%) 10022025 20425795 Nombre de photos taguées (types de déchets): 0 / 26 (0%) 10022025 20425795 Nombre de photos taguées (volume) : 0 / 26 (0%) elapsed_time : load_data_split_time_score 3.337860107421875e-06 elapsed_time : order_list_meta_photo_and_scores 7.62939453125e-06 ?????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0016601085662841797 elapsed_time : insert_dashboard_record_day_entry 0.028957366943359375 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423264 order by id desc limit 1 Qualite : 0.07650009788111799 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423265_11-02-2025_03_25_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423265 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423265 AND mptpi.`type`=4207 To do Qualite : 0.03082775628907242 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423266_11-02-2025_02_34_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423266 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423266 AND mptpi.`type`=4207 To do Qualite : 0.20770710642474585 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423267 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423267 AND mptpi.`type`=3594 To do Qualite : 0.46478871458569254 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423268_11-02-2025_00_42_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423268 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423268 AND mptpi.`type`=4200 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423269 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423270 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20424994_11-02-2025_03_00_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20424994 order by id desc limit 1 Qualite : 0.15089615373975407 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20424995_11-02-2025_03_30_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20424995 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20424995 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425791 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425792 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425793 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425794 order by id desc limit 1 Qualite : 0.08877007922187305 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425795_11-02-2025_03_34_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425795 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20425795 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425796 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425797 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425798 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425799 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425800 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425801 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425802 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425803 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425804 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425805 order by id desc limit 1 Qualite : 0.037925276526819866 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425806_11-02-2025_03_32_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425806 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20425806 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425807 order by id desc limit 1 Qualite : 0.15984814098589045 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20424997_11-02-2025_03_18_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20424997 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20424997 AND mptpi.`type`=3327 To do Qualite : 0.015606327247179095 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20424998_11-02-2025_02_23_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20424998 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20424998 AND mptpi.`type`=3327 To do Qualite : 0.23261406703885257 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20424999_11-02-2025_02_41_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20424999 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20424999 AND mptpi.`type`=3594 To do Qualite : 0.19914179581765348 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20424312_11-02-2025_01_42_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20424312 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20424312 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425000_11-02-2025_02_54_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425000 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425001_11-02-2025_02_31_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425001 order by id desc limit 1 Qualite : 0.2535467764217956 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425002_11-02-2025_02_29_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425002 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20425002 AND mptpi.`type`=3594 To do Qualite : 0.08557360668958641 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20425003_11-02-2025_03_20_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20425003 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20425003 AND mptpi.`type`=4207 To do Qualite : 0.04367358264311802 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423278_11-02-2025_01_44_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423278 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423278 AND mptpi.`type`=4207 To do Qualite : 0.05259447964667891 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423279_11-02-2025_01_55_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423279 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423279 AND mptpi.`type`=4207 To do Qualite : 0.04659670474030176 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423280_11-02-2025_01_58_16.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423280 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423280 AND mptpi.`type`=4207 To do Qualite : 0.08109472723835193 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423281_11-02-2025_00_24_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423281 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423281 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423282 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423283 order by id desc limit 1 Qualite : 0.18296439199232933 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423284_11-02-2025_00_27_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423284 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423284 AND mptpi.`type`=3594 To do Qualite : 0.05411750023929442 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423285_11-02-2025_01_17_32.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423285 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423285 AND mptpi.`type`=4211 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423286_11-02-2025_00_52_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423286 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423287 order by id desc limit 1 Qualite : 0.19405417404230213 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423288_11-02-2025_00_19_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423288 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423288 AND mptpi.`type`=3594 To do Qualite : 0.04755240840576223 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423289_11-02-2025_01_03_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423289 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423289 AND mptpi.`type`=4207 To do Qualite : 0.11405863253169997 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423290_11-02-2025_01_58_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423290 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423290 AND mptpi.`type`=4207 To do Qualite : 0.04685901461815405 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423291_11-02-2025_00_23_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423291 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423291 AND mptpi.`type`=4207 To do Qualite : 0.040974842720139536 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423292_11-02-2025_00_13_54.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423292 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423292 AND mptpi.`type`=4207 To do Qualite : 0.06698816144505071 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20423293_11-02-2025_00_18_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20423293 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20423293 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10022025': {'nb_upload': 26, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1336467271, 1336467269, 1336467265, 1336467262, 1336467258, 1336467218, 1336467216, 1336467168, 1336467164, 1336467153] Looping around the photos to save general results len do output : 1 /20425795Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467271', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467269', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467265', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467262', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467258', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467218', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467216', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467168', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467164', None, None, None, None, None, '2574456') ('3459', None, None, None, None, None, None, None, '2574456') ('3459', None, '1336467153', None, None, None, None, None, '2574456') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.01882791519165039 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.389764308929443 time spend to save output : 0.01905059814453125 total time spend for step 11 : 21.408814907073975 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 171.90user 148.56system 6:19.52elapsed 84%CPU (0avgtext+0avgdata 4361524maxresident)k 1985016inputs+182936outputs (8398major+14872403minor)pagefaults 0swaps