python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1868179 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 13>10 please execute split_portfolio.py -i 20193384 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2542528'] with mtr_portfolio_ids : ['20193384'] and first list_photo_ids : [] new path : /proc/1868179/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.8172848224639893 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Mon Feb 3 22:34:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10998 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-03 22:34:37.775206: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-03 22:34:37.807137: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-03 22:34:37.809088: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fdaac000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-03 22:34:37.809146: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-03 22:34:37.813475: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-03 22:34:38.153027: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2d6880e0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-03 22:34:38.153095: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-03 22:34:38.154596: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 22:34:38.156539: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 22:34:38.189483: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 22:34:38.209833: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 22:34:38.214518: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 22:34:38.247176: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 22:34:38.252547: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 22:34:38.311552: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 22:34:38.313481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 22:34:38.313947: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 22:34:38.315937: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-03 22:34:38.315961: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-03 22:34:38.315974: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-03 22:34:38.318152: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-03 22:34:38.663686: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 22:34:38.663841: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 22:34:38.663866: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 22:34:38.663889: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 22:34:38.663910: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 22:34:38.663930: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 22:34:38.663950: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 22:34:38.663971: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 22:34:38.665544: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 22:34:38.667103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-03 22:34:38.667150: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-03 22:34:38.667169: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 22:34:38.667187: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-03 22:34:38.667204: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-03 22:34:38.667221: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-03 22:34:38.667238: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-03 22:34:38.667255: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-03 22:34:38.668568: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-03 22:34:38.668604: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-03 22:34:38.668613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-03 22:34:38.668622: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-03 22:34:38.669952: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10193 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-03 22:34:47.382048: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-03 22:34:47.714598: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 30 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 33 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 11 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 34 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 6 Detection mask done ! Trying to reset tf kernel 1868432 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5706 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10998 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0021028518676757812 nb_pixel_total : 71121 time to create 1 rle with old method : 0.07310104370117188 length of segment : 369 time for calcul the mask position with numpy : 0.0009827613830566406 nb_pixel_total : 35800 time to create 1 rle with old method : 0.03638648986816406 length of segment : 552 time for calcul the mask position with numpy : 0.00022268295288085938 nb_pixel_total : 6692 time to create 1 rle with old method : 0.0071222782135009766 length of segment : 129 time for calcul the mask position with numpy : 0.00040912628173828125 nb_pixel_total : 23646 time to create 1 rle with old method : 0.024574995040893555 length of segment : 144 time for calcul the mask position with numpy : 0.00011229515075683594 nb_pixel_total : 3751 time to create 1 rle with old method : 0.0039637088775634766 length of segment : 117 time for calcul the mask position with numpy : 0.0008563995361328125 nb_pixel_total : 45186 time to create 1 rle with old method : 0.04658341407775879 length of segment : 222 time for calcul the mask position with numpy : 0.0005767345428466797 nb_pixel_total : 21263 time to create 1 rle with old method : 0.021978378295898438 length of segment : 288 time for calcul the mask position with numpy : 0.00014829635620117188 nb_pixel_total : 5263 time to create 1 rle with old method : 0.005667448043823242 length of segment : 97 time for calcul the mask position with numpy : 0.00013899803161621094 nb_pixel_total : 4358 time to create 1 rle with old method : 0.004664897918701172 length of segment : 82 time for calcul the mask position with numpy : 0.0010647773742675781 nb_pixel_total : 49319 time to create 1 rle with old method : 0.05069231986999512 length of segment : 276 time for calcul the mask position with numpy : 0.0030841827392578125 nb_pixel_total : 103773 time to create 1 rle with old method : 0.10289239883422852 length of segment : 521 time for calcul the mask position with numpy : 0.0005772113800048828 nb_pixel_total : 28910 time to create 1 rle with old method : 0.030101537704467773 length of segment : 267 time for calcul the mask position with numpy : 0.0015459060668945312 nb_pixel_total : 77201 time to create 1 rle with old method : 0.07911825180053711 length of segment : 416 time for calcul the mask position with numpy : 0.0011305809020996094 nb_pixel_total : 56314 time to create 1 rle with old method : 0.05825972557067871 length of segment : 292 time for calcul the mask position with numpy : 0.00012183189392089844 nb_pixel_total : 1569 time to create 1 rle with old method : 0.0018956661224365234 length of segment : 48 time for calcul the mask position with numpy : 0.0006322860717773438 nb_pixel_total : 28144 time to create 1 rle with old method : 0.029677867889404297 length of segment : 377 time for calcul the mask position with numpy : 0.0002200603485107422 nb_pixel_total : 6988 time to create 1 rle with old method : 0.008042097091674805 length of segment : 82 time for calcul the mask position with numpy : 0.0003437995910644531 nb_pixel_total : 10101 time to create 1 rle with old method : 0.011640548706054688 length of segment : 142 time for calcul the mask position with numpy : 8.821487426757812e-05 nb_pixel_total : 1367 time to create 1 rle with old method : 0.001581430435180664 length of segment : 46 time for calcul the mask position with numpy : 0.0011835098266601562 nb_pixel_total : 64563 time to create 1 rle with old method : 0.06619739532470703 length of segment : 349 time for calcul the mask position with numpy : 0.0006971359252929688 nb_pixel_total : 19595 time to create 1 rle with old method : 0.02132582664489746 length of segment : 281 time for calcul the mask position with numpy : 0.00026679039001464844 nb_pixel_total : 10957 time to create 1 rle with old method : 0.011988162994384766 length of segment : 170 time for calcul the mask position with numpy : 0.006497859954833984 nb_pixel_total : 284515 time to create 1 rle with new method : 0.01959538459777832 length of segment : 677 time for calcul the mask position with numpy : 0.0004019737243652344 nb_pixel_total : 12596 time to create 1 rle with old method : 0.013476371765136719 length of segment : 133 time for calcul the mask position with numpy : 0.00013828277587890625 nb_pixel_total : 4192 time to create 1 rle with old method : 0.004774808883666992 length of segment : 61 time for calcul the mask position with numpy : 0.0018978118896484375 nb_pixel_total : 15557 time to create 1 rle with old method : 0.01643538475036621 length of segment : 172 time for calcul the mask position with numpy : 0.0011582374572753906 nb_pixel_total : 32179 time to create 1 rle with old method : 0.034662723541259766 length of segment : 465 time for calcul the mask position with numpy : 0.0011582374572753906 nb_pixel_total : 56334 time to create 1 rle with old method : 0.06261324882507324 length of segment : 404 time for calcul the mask position with numpy : 0.005480289459228516 nb_pixel_total : 266714 time to create 1 rle with new method : 0.02258777618408203 length of segment : 701 time for calcul the mask position with numpy : 0.0023496150970458984 nb_pixel_total : 105042 time to create 1 rle with old method : 0.11034846305847168 length of segment : 399 time for calcul the mask position with numpy : 0.0003771781921386719 nb_pixel_total : 10277 time to create 1 rle with old method : 0.011189460754394531 length of segment : 177 time for calcul the mask position with numpy : 0.0006864070892333984 nb_pixel_total : 31026 time to create 1 rle with old method : 0.032329559326171875 length of segment : 486 time for calcul the mask position with numpy : 0.0005197525024414062 nb_pixel_total : 20995 time to create 1 rle with old method : 0.021921396255493164 length of segment : 222 time for calcul the mask position with numpy : 0.00020360946655273438 nb_pixel_total : 8131 time to create 1 rle with old method : 0.009151458740234375 length of segment : 128 time for calcul the mask position with numpy : 0.00027060508728027344 nb_pixel_total : 12389 time to create 1 rle with old method : 0.013827323913574219 length of segment : 124 time for calcul the mask position with numpy : 0.002351522445678711 nb_pixel_total : 123900 time to create 1 rle with old method : 0.127824068069458 length of segment : 556 time for calcul the mask position with numpy : 0.0010826587677001953 nb_pixel_total : 49764 time to create 1 rle with old method : 0.05066847801208496 length of segment : 382 time for calcul the mask position with numpy : 0.0006494522094726562 nb_pixel_total : 38364 time to create 1 rle with old method : 0.04124093055725098 length of segment : 250 time for calcul the mask position with numpy : 0.0005900859832763672 nb_pixel_total : 29814 time to create 1 rle with old method : 0.03452467918395996 length of segment : 170 time for calcul the mask position with numpy : 0.0028412342071533203 nb_pixel_total : 146299 time to create 1 rle with old method : 0.15146780014038086 length of segment : 651 time for calcul the mask position with numpy : 0.0006194114685058594 nb_pixel_total : 25677 time to create 1 rle with old method : 0.028805255889892578 length of segment : 179 time for calcul the mask position with numpy : 0.0012254714965820312 nb_pixel_total : 32782 time to create 1 rle with old method : 0.03703045845031738 length of segment : 363 time for calcul the mask position with numpy : 0.00013184547424316406 nb_pixel_total : 2135 time to create 1 rle with old method : 0.0026597976684570312 length of segment : 61 time for calcul the mask position with numpy : 0.0008895397186279297 nb_pixel_total : 34603 time to create 1 rle with old method : 0.03703665733337402 length of segment : 355 time for calcul the mask position with numpy : 9.655952453613281e-05 nb_pixel_total : 1273 time to create 1 rle with old method : 0.0015683174133300781 length of segment : 56 time for calcul the mask position with numpy : 0.0007541179656982422 nb_pixel_total : 35057 time to create 1 rle with old method : 0.0422060489654541 length of segment : 346 time for calcul the mask position with numpy : 0.002836942672729492 nb_pixel_total : 43907 time to create 1 rle with old method : 0.045267581939697266 length of segment : 417 time for calcul the mask position with numpy : 0.00029921531677246094 nb_pixel_total : 3527 time to create 1 rle with old method : 0.0038275718688964844 length of segment : 83 time for calcul the mask position with numpy : 0.0016241073608398438 nb_pixel_total : 32215 time to create 1 rle with old method : 0.03342914581298828 length of segment : 329 time for calcul the mask position with numpy : 0.0007886886596679688 nb_pixel_total : 11545 time to create 1 rle with old method : 0.012567758560180664 length of segment : 182 time for calcul the mask position with numpy : 0.0021584033966064453 nb_pixel_total : 65979 time to create 1 rle with old method : 0.06800723075866699 length of segment : 296 time for calcul the mask position with numpy : 0.0011544227600097656 nb_pixel_total : 33092 time to create 1 rle with old method : 0.03446006774902344 length of segment : 172 time for calcul the mask position with numpy : 0.013528108596801758 nb_pixel_total : 104070 time to create 1 rle with old method : 0.10762143135070801 length of segment : 622 time for calcul the mask position with numpy : 0.0017163753509521484 nb_pixel_total : 27360 time to create 1 rle with old method : 0.028506994247436523 length of segment : 192 time for calcul the mask position with numpy : 0.0008187294006347656 nb_pixel_total : 30311 time to create 1 rle with old method : 0.030394315719604492 length of segment : 340 time for calcul the mask position with numpy : 0.0017185211181640625 nb_pixel_total : 41021 time to create 1 rle with old method : 0.04215097427368164 length of segment : 371 time for calcul the mask position with numpy : 0.0005576610565185547 nb_pixel_total : 24881 time to create 1 rle with old method : 0.02567458152770996 length of segment : 221 time for calcul the mask position with numpy : 0.0010266304016113281 nb_pixel_total : 11385 time to create 1 rle with old method : 0.012703657150268555 length of segment : 233 time for calcul the mask position with numpy : 0.0011472702026367188 nb_pixel_total : 18877 time to create 1 rle with old method : 0.021251916885375977 length of segment : 164 time for calcul the mask position with numpy : 0.0034782886505126953 nb_pixel_total : 155357 time to create 1 rle with new method : 0.012050151824951172 length of segment : 689 time for calcul the mask position with numpy : 0.0022399425506591797 nb_pixel_total : 139194 time to create 1 rle with old method : 0.1523914337158203 length of segment : 337 time for calcul the mask position with numpy : 0.0005562305450439453 nb_pixel_total : 33507 time to create 1 rle with old method : 0.035463809967041016 length of segment : 196 time for calcul the mask position with numpy : 0.0016524791717529297 nb_pixel_total : 107136 time to create 1 rle with old method : 0.10955238342285156 length of segment : 452 time for calcul the mask position with numpy : 0.030368328094482422 nb_pixel_total : 1800950 time to create 1 rle with new method : 0.060889482498168945 length of segment : 1889 time for calcul the mask position with numpy : 0.0003311634063720703 nb_pixel_total : 14393 time to create 1 rle with old method : 0.015042543411254883 length of segment : 233 time for calcul the mask position with numpy : 0.04635429382324219 nb_pixel_total : 2294004 time to create 1 rle with new method : 0.12289214134216309 length of segment : 3436 time for calcul the mask position with numpy : 0.0043201446533203125 nb_pixel_total : 245001 time to create 1 rle with new method : 0.00626826286315918 length of segment : 556 time for calcul the mask position with numpy : 0.003609895706176758 nb_pixel_total : 167384 time to create 1 rle with new method : 0.005693912506103516 length of segment : 741 time for calcul the mask position with numpy : 0.0036764144897460938 nb_pixel_total : 219388 time to create 1 rle with new method : 0.00829458236694336 length of segment : 1482 time for calcul the mask position with numpy : 0.010869741439819336 nb_pixel_total : 555401 time to create 1 rle with new method : 0.02312302589416504 length of segment : 1410 time for calcul the mask position with numpy : 0.0040912628173828125 nb_pixel_total : 152843 time to create 1 rle with new method : 0.005517005920410156 length of segment : 426 time for calcul the mask position with numpy : 0.0016858577728271484 nb_pixel_total : 45442 time to create 1 rle with old method : 0.04682779312133789 length of segment : 398 time for calcul the mask position with numpy : 0.0018532276153564453 nb_pixel_total : 54634 time to create 1 rle with old method : 0.05671525001525879 length of segment : 509 time for calcul the mask position with numpy : 0.0037517547607421875 nb_pixel_total : 151452 time to create 1 rle with new method : 0.0066797733306884766 length of segment : 857 time for calcul the mask position with numpy : 0.004458904266357422 nb_pixel_total : 131162 time to create 1 rle with old method : 0.13283276557922363 length of segment : 758 time for calcul the mask position with numpy : 0.0012462139129638672 nb_pixel_total : 38644 time to create 1 rle with old method : 0.04131793975830078 length of segment : 261 time for calcul the mask position with numpy : 0.0013921260833740234 nb_pixel_total : 49512 time to create 1 rle with old method : 0.05285072326660156 length of segment : 217 time for calcul the mask position with numpy : 0.004247426986694336 nb_pixel_total : 163291 time to create 1 rle with new method : 0.007375001907348633 length of segment : 436 time for calcul the mask position with numpy : 0.0030584335327148438 nb_pixel_total : 84627 time to create 1 rle with old method : 0.09504961967468262 length of segment : 496 time for calcul the mask position with numpy : 0.0004200935363769531 nb_pixel_total : 11703 time to create 1 rle with old method : 0.013289928436279297 length of segment : 90 time for calcul the mask position with numpy : 7.009506225585938e-05 nb_pixel_total : 1587 time to create 1 rle with old method : 0.0019125938415527344 length of segment : 58 time for calcul the mask position with numpy : 0.001848459243774414 nb_pixel_total : 83583 time to create 1 rle with old method : 0.08679437637329102 length of segment : 427 time for calcul the mask position with numpy : 0.001005411148071289 nb_pixel_total : 18334 time to create 1 rle with old method : 0.019454240798950195 length of segment : 309 time for calcul the mask position with numpy : 0.001653909683227539 nb_pixel_total : 50896 time to create 1 rle with old method : 0.05204439163208008 length of segment : 523 time for calcul the mask position with numpy : 0.004019498825073242 nb_pixel_total : 101254 time to create 1 rle with old method : 0.10172629356384277 length of segment : 696 time for calcul the mask position with numpy : 0.002445220947265625 nb_pixel_total : 77775 time to create 1 rle with old method : 0.07847476005554199 length of segment : 364 time for calcul the mask position with numpy : 0.0014069080352783203 nb_pixel_total : 50358 time to create 1 rle with old method : 0.05054593086242676 length of segment : 458 time for calcul the mask position with numpy : 0.0002875328063964844 nb_pixel_total : 3134 time to create 1 rle with old method : 0.003485441207885742 length of segment : 115 time for calcul the mask position with numpy : 0.007672786712646484 nb_pixel_total : 193162 time to create 1 rle with new method : 0.012157917022705078 length of segment : 560 time for calcul the mask position with numpy : 0.0006215572357177734 nb_pixel_total : 26559 time to create 1 rle with old method : 0.027360916137695312 length of segment : 202 time for calcul the mask position with numpy : 0.011028528213500977 nb_pixel_total : 439794 time to create 1 rle with new method : 0.022185087203979492 length of segment : 941 time for calcul the mask position with numpy : 0.0015735626220703125 nb_pixel_total : 87351 time to create 1 rle with old method : 0.0873560905456543 length of segment : 388 time for calcul the mask position with numpy : 0.003425121307373047 nb_pixel_total : 104548 time to create 1 rle with old method : 0.10817670822143555 length of segment : 902 time for calcul the mask position with numpy : 0.0005877017974853516 nb_pixel_total : 23885 time to create 1 rle with old method : 0.024266958236694336 length of segment : 299 time for calcul the mask position with numpy : 0.0007338523864746094 nb_pixel_total : 24810 time to create 1 rle with old method : 0.027190208435058594 length of segment : 312 time for calcul the mask position with numpy : 0.00020933151245117188 nb_pixel_total : 3344 time to create 1 rle with old method : 0.0037169456481933594 length of segment : 77 time for calcul the mask position with numpy : 0.004372119903564453 nb_pixel_total : 158284 time to create 1 rle with new method : 0.008069515228271484 length of segment : 880 time for calcul the mask position with numpy : 0.05853128433227539 nb_pixel_total : 3057985 time to create 1 rle with new method : 0.10022091865539551 length of segment : 4078 time for calcul the mask position with numpy : 0.0002474784851074219 nb_pixel_total : 14009 time to create 1 rle with old method : 0.014607667922973633 length of segment : 140 time for calcul the mask position with numpy : 0.0007238388061523438 nb_pixel_total : 35211 time to create 1 rle with old method : 0.035906076431274414 length of segment : 273 time for calcul the mask position with numpy : 0.002299070358276367 nb_pixel_total : 176720 time to create 1 rle with new method : 0.006042003631591797 length of segment : 563 time for calcul the mask position with numpy : 0.00026869773864746094 nb_pixel_total : 8795 time to create 1 rle with old method : 0.01136636734008789 length of segment : 108 time for calcul the mask position with numpy : 0.0011477470397949219 nb_pixel_total : 73209 time to create 1 rle with old method : 0.07360982894897461 length of segment : 393 time for calcul the mask position with numpy : 0.00018286705017089844 nb_pixel_total : 2342 time to create 1 rle with old method : 0.002644062042236328 length of segment : 88 time for calcul the mask position with numpy : 0.0032074451446533203 nb_pixel_total : 124857 time to create 1 rle with old method : 0.12685275077819824 length of segment : 317 time for calcul the mask position with numpy : 0.0003325939178466797 nb_pixel_total : 11054 time to create 1 rle with old method : 0.012096166610717773 length of segment : 107 time for calcul the mask position with numpy : 0.0034074783325195312 nb_pixel_total : 147844 time to create 1 rle with old method : 0.15845775604248047 length of segment : 444 time for calcul the mask position with numpy : 0.00022172927856445312 nb_pixel_total : 5201 time to create 1 rle with old method : 0.00621795654296875 length of segment : 69 time for calcul the mask position with numpy : 0.00022912025451660156 nb_pixel_total : 6964 time to create 1 rle with old method : 0.007802009582519531 length of segment : 130 time for calcul the mask position with numpy : 0.003192901611328125 nb_pixel_total : 145086 time to create 1 rle with old method : 0.1525895595550537 length of segment : 518 time for calcul the mask position with numpy : 0.0002474784851074219 nb_pixel_total : 9957 time to create 1 rle with old method : 0.01143336296081543 length of segment : 99 time for calcul the mask position with numpy : 0.0013592243194580078 nb_pixel_total : 68174 time to create 1 rle with old method : 0.07089543342590332 length of segment : 357 time for calcul the mask position with numpy : 0.0004143714904785156 nb_pixel_total : 2283 time to create 1 rle with old method : 0.0031821727752685547 length of segment : 114 time for calcul the mask position with numpy : 0.0001964569091796875 nb_pixel_total : 3432 time to create 1 rle with old method : 0.004259586334228516 length of segment : 71 time for calcul the mask position with numpy : 0.0016760826110839844 nb_pixel_total : 79388 time to create 1 rle with old method : 0.08291935920715332 length of segment : 462 time for calcul the mask position with numpy : 0.0037162303924560547 nb_pixel_total : 182729 time to create 1 rle with new method : 0.006536722183227539 length of segment : 575 time for calcul the mask position with numpy : 0.004774332046508789 nb_pixel_total : 214680 time to create 1 rle with new method : 0.005848407745361328 length of segment : 407 time for calcul the mask position with numpy : 0.00041675567626953125 nb_pixel_total : 10682 time to create 1 rle with old method : 0.012192487716674805 length of segment : 89 time for calcul the mask position with numpy : 0.00325775146484375 nb_pixel_total : 103906 time to create 1 rle with old method : 0.10738348960876465 length of segment : 248 time for calcul the mask position with numpy : 0.0002994537353515625 nb_pixel_total : 6983 time to create 1 rle with old method : 0.007911205291748047 length of segment : 91 time for calcul the mask position with numpy : 0.0014121532440185547 nb_pixel_total : 29220 time to create 1 rle with old method : 0.030711650848388672 length of segment : 215 time for calcul the mask position with numpy : 0.0003237724304199219 nb_pixel_total : 5903 time to create 1 rle with old method : 0.006883144378662109 length of segment : 101 time for calcul the mask position with numpy : 0.002214193344116211 nb_pixel_total : 25555 time to create 1 rle with old method : 0.028083086013793945 length of segment : 223 time for calcul the mask position with numpy : 0.0007517337799072266 nb_pixel_total : 20441 time to create 1 rle with old method : 0.021519899368286133 length of segment : 176 time for calcul the mask position with numpy : 0.0028395652770996094 nb_pixel_total : 106214 time to create 1 rle with old method : 0.1092519760131836 length of segment : 378 time for calcul the mask position with numpy : 0.0003223419189453125 nb_pixel_total : 12601 time to create 1 rle with old method : 0.01477503776550293 length of segment : 125 time for calcul the mask position with numpy : 0.0013828277587890625 nb_pixel_total : 59389 time to create 1 rle with old method : 0.06116986274719238 length of segment : 364 time for calcul the mask position with numpy : 0.044133901596069336 nb_pixel_total : 1054309 time to create 1 rle with new method : 0.07592058181762695 length of segment : 2458 time for calcul the mask position with numpy : 0.0022172927856445312 nb_pixel_total : 88321 time to create 1 rle with old method : 0.08937644958496094 length of segment : 415 time for calcul the mask position with numpy : 0.001062631607055664 nb_pixel_total : 35863 time to create 1 rle with old method : 0.03686642646789551 length of segment : 259 time for calcul the mask position with numpy : 0.004091739654541016 nb_pixel_total : 117068 time to create 1 rle with old method : 0.11900925636291504 length of segment : 465 time for calcul the mask position with numpy : 0.002826213836669922 nb_pixel_total : 106222 time to create 1 rle with old method : 0.10964655876159668 length of segment : 677 time for calcul the mask position with numpy : 0.00014853477478027344 nb_pixel_total : 2939 time to create 1 rle with old method : 0.0033431053161621094 length of segment : 83 time for calcul the mask position with numpy : 0.011570930480957031 nb_pixel_total : 780655 time to create 1 rle with new method : 0.02422952651977539 length of segment : 1502 time for calcul the mask position with numpy : 0.0012869834899902344 nb_pixel_total : 64602 time to create 1 rle with old method : 0.0660405158996582 length of segment : 378 time for calcul the mask position with numpy : 0.0004253387451171875 nb_pixel_total : 14892 time to create 1 rle with old method : 0.016398906707763672 length of segment : 180 time for calcul the mask position with numpy : 0.001535177230834961 nb_pixel_total : 92917 time to create 1 rle with old method : 0.09584975242614746 length of segment : 388 time for calcul the mask position with numpy : 0.00011157989501953125 nb_pixel_total : 3331 time to create 1 rle with old method : 0.003664255142211914 length of segment : 68 time for calcul the mask position with numpy : 0.00972127914428711 nb_pixel_total : 508649 time to create 1 rle with new method : 0.023655414581298828 length of segment : 1330 time for calcul the mask position with numpy : 0.008875131607055664 nb_pixel_total : 488484 time to create 1 rle with new method : 0.02373027801513672 length of segment : 1160 time for calcul the mask position with numpy : 0.0031147003173828125 nb_pixel_total : 194182 time to create 1 rle with new method : 0.007705211639404297 length of segment : 574 time for calcul the mask position with numpy : 0.0017752647399902344 nb_pixel_total : 29479 time to create 1 rle with old method : 0.03163409233093262 length of segment : 679 time for calcul the mask position with numpy : 0.00441741943359375 nb_pixel_total : 131373 time to create 1 rle with old method : 0.1475234031677246 length of segment : 447 time for calcul the mask position with numpy : 0.0035698413848876953 nb_pixel_total : 125088 time to create 1 rle with old method : 0.14081263542175293 length of segment : 479 time for calcul the mask position with numpy : 0.004537105560302734 nb_pixel_total : 248177 time to create 1 rle with new method : 0.010807514190673828 length of segment : 804 time for calcul the mask position with numpy : 0.0014700889587402344 nb_pixel_total : 93913 time to create 1 rle with old method : 0.09464502334594727 length of segment : 400 time for calcul the mask position with numpy : 0.005186319351196289 nb_pixel_total : 336152 time to create 1 rle with new method : 0.011185884475708008 length of segment : 732 time for calcul the mask position with numpy : 0.0004703998565673828 nb_pixel_total : 17414 time to create 1 rle with old method : 0.0193789005279541 length of segment : 107 time for calcul the mask position with numpy : 0.0011234283447265625 nb_pixel_total : 29091 time to create 1 rle with old method : 0.03005242347717285 length of segment : 336 time for calcul the mask position with numpy : 0.0004150867462158203 nb_pixel_total : 10724 time to create 1 rle with old method : 0.01167750358581543 length of segment : 124 time for calcul the mask position with numpy : 0.0004124641418457031 nb_pixel_total : 13525 time to create 1 rle with old method : 0.014690399169921875 length of segment : 126 time for calcul the mask position with numpy : 0.00025010108947753906 nb_pixel_total : 4653 time to create 1 rle with old method : 0.004929780960083008 length of segment : 124 time for calcul the mask position with numpy : 0.0002562999725341797 nb_pixel_total : 7230 time to create 1 rle with old method : 0.008198261260986328 length of segment : 72 time for calcul the mask position with numpy : 0.0007436275482177734 nb_pixel_total : 19304 time to create 1 rle with old method : 0.03187131881713867 length of segment : 145 time for calcul the mask position with numpy : 0.0024421215057373047 nb_pixel_total : 83103 time to create 1 rle with old method : 0.08317947387695312 length of segment : 569 time for calcul the mask position with numpy : 0.0009391307830810547 nb_pixel_total : 13576 time to create 1 rle with old method : 0.014275312423706055 length of segment : 220 time for calcul the mask position with numpy : 0.0003459453582763672 nb_pixel_total : 8923 time to create 1 rle with old method : 0.010157585144042969 length of segment : 131 time for calcul the mask position with numpy : 0.00892329216003418 nb_pixel_total : 371164 time to create 1 rle with new method : 0.013905763626098633 length of segment : 736 time for calcul the mask position with numpy : 0.004227399826049805 nb_pixel_total : 147537 time to create 1 rle with old method : 0.14955854415893555 length of segment : 677 time for calcul the mask position with numpy : 0.0018515586853027344 nb_pixel_total : 63717 time to create 1 rle with old method : 0.06499886512756348 length of segment : 278 time for calcul the mask position with numpy : 9.870529174804688e-05 nb_pixel_total : 2399 time to create 1 rle with old method : 0.002639293670654297 length of segment : 96 time for calcul the mask position with numpy : 0.0012011528015136719 nb_pixel_total : 6340 time to create 1 rle with old method : 0.007338047027587891 length of segment : 119 time for calcul the mask position with numpy : 0.00048089027404785156 nb_pixel_total : 17760 time to create 1 rle with old method : 0.018276453018188477 length of segment : 159 time for calcul the mask position with numpy : 0.0011734962463378906 nb_pixel_total : 32706 time to create 1 rle with old method : 0.033983469009399414 length of segment : 370 time for calcul the mask position with numpy : 0.00047707557678222656 nb_pixel_total : 13909 time to create 1 rle with old method : 0.01565718650817871 length of segment : 132 time for calcul the mask position with numpy : 0.0007374286651611328 nb_pixel_total : 26943 time to create 1 rle with old method : 0.02775430679321289 length of segment : 203 time for calcul the mask position with numpy : 0.0008845329284667969 nb_pixel_total : 29131 time to create 1 rle with old method : 0.031388044357299805 length of segment : 187 time for calcul the mask position with numpy : 0.00017070770263671875 nb_pixel_total : 2546 time to create 1 rle with old method : 0.00292205810546875 length of segment : 64 time for calcul the mask position with numpy : 0.00315093994140625 nb_pixel_total : 77764 time to create 1 rle with old method : 0.08624815940856934 length of segment : 540 time for calcul the mask position with numpy : 0.0003635883331298828 nb_pixel_total : 5162 time to create 1 rle with old method : 0.008832216262817383 length of segment : 60 time for calcul the mask position with numpy : 0.0025014877319335938 nb_pixel_total : 61265 time to create 1 rle with old method : 0.0725412368774414 length of segment : 322 time for calcul the mask position with numpy : 0.00028586387634277344 nb_pixel_total : 5136 time to create 1 rle with old method : 0.006350040435791016 length of segment : 80 time for calcul the mask position with numpy : 0.0003921985626220703 nb_pixel_total : 10023 time to create 1 rle with old method : 0.01168513298034668 length of segment : 104 time for calcul the mask position with numpy : 0.0028853416442871094 nb_pixel_total : 56483 time to create 1 rle with old method : 0.0768439769744873 length of segment : 264 time for calcul the mask position with numpy : 0.008360862731933594 nb_pixel_total : 308262 time to create 1 rle with new method : 0.012693643569946289 length of segment : 670 time for calcul the mask position with numpy : 0.0006806850433349609 nb_pixel_total : 12918 time to create 1 rle with old method : 0.022800207138061523 length of segment : 157 time for calcul the mask position with numpy : 0.0001888275146484375 nb_pixel_total : 1251 time to create 1 rle with old method : 0.0016002655029296875 length of segment : 57 time for calcul the mask position with numpy : 0.00024771690368652344 nb_pixel_total : 8482 time to create 1 rle with old method : 0.010134220123291016 length of segment : 77 time for calcul the mask position with numpy : 0.06312417984008789 nb_pixel_total : 3388076 time to create 1 rle with new method : 0.10440659523010254 length of segment : 3378 time for calcul the mask position with numpy : 0.010680198669433594 nb_pixel_total : 709818 time to create 1 rle with new method : 0.026885032653808594 length of segment : 1231 time for calcul the mask position with numpy : 0.0004608631134033203 nb_pixel_total : 32155 time to create 1 rle with old method : 0.03432130813598633 length of segment : 157 time for calcul the mask position with numpy : 0.0043392181396484375 nb_pixel_total : 364461 time to create 1 rle with new method : 0.01011967658996582 length of segment : 630 time for calcul the mask position with numpy : 0.0007700920104980469 nb_pixel_total : 34321 time to create 1 rle with old method : 0.037068843841552734 length of segment : 330 time for calcul the mask position with numpy : 0.0005619525909423828 nb_pixel_total : 35389 time to create 1 rle with old method : 0.037244319915771484 length of segment : 369 time spent for convertir_results : 19.672955751419067 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 184 chid ids of type : 3663 Number RLEs to save : 77989 save missing photos in datou_result : time spend for datou_step_exec : 53.077184438705444 time spend to save output : 5.216146469116211 total time spend for step 1 : 58.293330907821655 step2:crop_condition Mon Feb 3 22:35:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 184 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 39 About to insert : list_path_to_insert length 39 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 44 About to insert : list_path_to_insert length 44 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1334287114, 1334287106, 1334287082, 1334287033, 1334286966, 1334285826, 1334285724, 1334285397, 1334285394, 1334285376] Looping around the photos to save general results len do output : 106 /-3652354353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354392Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354398Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354407Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354427Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354434Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354425Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354462Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354359Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354396Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354479Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354464Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354378Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354399Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354409Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354402Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354408Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354421Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354437Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354436Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354468Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354489Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354514Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354511Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354501Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354523Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354528Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354519Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354534Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354502Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354380Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354533Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3652354446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287114', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287106', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287082', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287033', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334286966', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285826', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285724', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285397', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285394', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285376', None, None, None, None, None, '2542528') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 328 time used for this insertion : 0.03792548179626465 save_final save missing photos in datou_result : time spend for datou_step_exec : 44.82101798057556 time spend to save output : 0.040338993072509766 total time spend for step 2 : 44.86135697364807 step3:thcl Mon Feb 3 22:36:16 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.006392478942871094 time to convert the images to numpy array : 0.06713747978210449 time to import caffe and check if the image exist : 0.011972188949584961 time to convert the images to numpy array : 0.14849090576171875 time to import caffe and check if the image exist : 0.012194156646728516 time to convert the images to numpy array : 0.1568315029144287 time to import caffe and check if the image exist : 0.009058952331542969 time to convert the images to numpy array : 0.16457080841064453 time to import caffe and check if the image exist : 0.00970005989074707 time to convert the images to numpy array : 0.16707777976989746 time to import caffe and check if the image exist : 0.009577035903930664 time to convert the images to numpy array : 0.16837573051452637 time to import caffe and check if the image exist : 0.008763790130615234 time to convert the images to numpy array : 0.1740877628326416 time to import caffe and check if the image exist : 0.01304006576538086 time to convert the images to numpy array : 0.17213749885559082 time to import caffe and check if the image exist : 0.008641481399536133 time to convert the images to numpy array : 0.1777794361114502 time to import caffe and check if the image exist : 0.011101245880126953 time to convert the images to numpy array : 0.2344982624053955 total time to convert the images to numpy array : 0.4256420135498047 list photo_ids error: [] list photo_ids correct : [-3652354525, -3652354380, -3652354366, -3652354455, -3652354452, -3652354533, -3652354446, -3652354489, -3652354487, -3652354494, -3652354476, -3652354475, -3652354514, -3652354506, -3652354508, -3652354507, -3652354511, -3652354501, -3652354406, -3652354398, -3652354403, -3652354407, -3652354413, -3652354414, -3652354415, -3652354428, -3652354427, -3652354438, -3652354431, -3652354523, -3652354528, -3652354526, -3652354527, -3652354519, -3652354534, -3652354370, -3652354435, -3652354474, -3652354502, -3652354517, -3652354434, -3652354423, -3652354425, -3652354426, -3652354424, -3652354447, -3652354461, -3652354459, -3652354457, -3652354462, -3652354469, -3652354353, -3652354354, -3652354364, -3652354385, -3652354381, -3652354405, -3652354391, -3652354382, -3652354393, -3652354389, -3652354392, -3652354399, -3652354409, -3652354412, -3652354402, -3652354388, -3652354394, -3652354384, -3652354408, -3652354401, -3652354421, -3652354437, -3652354477, -3652354481, -3652354471, -3652354470, -3652354483, -3652354518, -3652354359, -3652354396, -3652354479, -3652354498, -3652354375, -3652354365, -3652354444, -3652354464, -3652354497, -3652354496, -3652354512, -3652354363, -3652354378, -3652354362, -3652354368, -3652354372, -3652354429, -3652354436, -3652354445, -3652354449, -3652354468, -3652354467, -3652354499, -3652354492, -3652354473, -3652354484, -3652354482] number of photos to traite : 106 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10774 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10336 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.2362463474273682 time used to do the prediction : 0.1907029151916504 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10332 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.8699848651885986 time used to do the prediction : 0.2008829116821289 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.002716064453125e-05 save missing photos in datou_result : time spend for datou_step_exec : 10.880972862243652 time spend to save output : 0.0011246204376220703 total time spend for step 3 : 10.882097482681274 step4:merge_mask_thcl_custom Mon Feb 3 22:36:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 184 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1334287114, 1334287106, 1334287082, 1334287033, 1334286966, 1334285826, 1334285724, 1334285397, 1334285394, 1334285376] Looping around the photos to save general results len do output : 10 /1334287114Didn't retrieve data .Didn't retrieve data . /1334287106Didn't retrieve data .Didn't retrieve data . /1334287082Didn't retrieve data .Didn't retrieve data . /1334287033Didn't retrieve data .Didn't retrieve data . /1334286966Didn't retrieve data .Didn't retrieve data . /1334285826Didn't retrieve data .Didn't retrieve data . /1334285724Didn't retrieve data .Didn't retrieve data . /1334285397Didn't retrieve data .Didn't retrieve data . /1334285394Didn't retrieve data .Didn't retrieve data . /1334285376Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287114', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287106', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287082', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287033', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334286966', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285826', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285724', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285397', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285394', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285376', None, None, None, None, None, '2542528') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013045549392700195 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06592249870300293 time spend to save output : 0.013499259948730469 total time spend for step 4 : 0.0794217586517334 step5:rle_unique_nms_with_priority Mon Feb 3 22:36:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 28 nb_hashtags : 8 time to prepare the origin masks : 10.482795000076294 time for calcul the mask position with numpy : 0.08888030052185059 nb_pixel_total : 5971967 time to create 1 rle with new method : 0.14907097816467285 time for calcul the mask position with numpy : 0.029372692108154297 nb_pixel_total : 21207 time to create 1 rle with old method : 0.023268699645996094 time for calcul the mask position with numpy : 0.02895498275756836 nb_pixel_total : 4169 time to create 1 rle with old method : 0.004682064056396484 time for calcul the mask position with numpy : 0.030655860900878906 nb_pixel_total : 103213 time to create 1 rle with old method : 0.11322760581970215 time for calcul the mask position with numpy : 0.028972387313842773 nb_pixel_total : 6667 time to create 1 rle with old method : 0.007529497146606445 time for calcul the mask position with numpy : 0.029021263122558594 nb_pixel_total : 35769 time to create 1 rle with old method : 0.04432559013366699 time for calcul the mask position with numpy : 0.033611297607421875 nb_pixel_total : 5247 time to create 1 rle with old method : 0.008289575576782227 time for calcul the mask position with numpy : 0.03415822982788086 nb_pixel_total : 32170 time to create 1 rle with old method : 0.04797220230102539 time for calcul the mask position with numpy : 0.034653425216674805 nb_pixel_total : 6900 time to create 1 rle with old method : 0.011144876480102539 time for calcul the mask position with numpy : 0.03899979591369629 nb_pixel_total : 70824 time to create 1 rle with old method : 0.08315587043762207 time for calcul the mask position with numpy : 0.03910493850708008 nb_pixel_total : 283821 time to create 1 rle with new method : 0.14983558654785156 time for calcul the mask position with numpy : 0.02933955192565918 nb_pixel_total : 10072 time to create 1 rle with old method : 0.011070013046264648 time for calcul the mask position with numpy : 0.031360626220703125 nb_pixel_total : 76771 time to create 1 rle with old method : 0.09269428253173828 time for calcul the mask position with numpy : 0.030142545700073242 nb_pixel_total : 56140 time to create 1 rle with old method : 0.06027078628540039 time for calcul the mask position with numpy : 0.02986764907836914 nb_pixel_total : 56128 time to create 1 rle with old method : 0.06911826133728027 time for calcul the mask position with numpy : 0.028957128524780273 nb_pixel_total : 15430 time to create 1 rle with old method : 0.01713418960571289 time for calcul the mask position with numpy : 0.029545068740844727 nb_pixel_total : 28485 time to create 1 rle with old method : 0.031778812408447266 time for calcul the mask position with numpy : 0.029211759567260742 nb_pixel_total : 1564 time to create 1 rle with old method : 0.002002716064453125 time for calcul the mask position with numpy : 0.029764652252197266 nb_pixel_total : 44869 time to create 1 rle with old method : 0.049041032791137695 time for calcul the mask position with numpy : 0.02934575080871582 nb_pixel_total : 4339 time to create 1 rle with old method : 0.0046312808990478516 time for calcul the mask position with numpy : 0.03052687644958496 nb_pixel_total : 1359 time to create 1 rle with old method : 0.0016009807586669922 time for calcul the mask position with numpy : 0.029346466064453125 nb_pixel_total : 64743 time to create 1 rle with old method : 0.10039591789245605 time for calcul the mask position with numpy : 0.037960052490234375 nb_pixel_total : 49204 time to create 1 rle with old method : 0.05300474166870117 time for calcul the mask position with numpy : 0.029624462127685547 nb_pixel_total : 3745 time to create 1 rle with old method : 0.0044078826904296875 time for calcul the mask position with numpy : 0.0291445255279541 nb_pixel_total : 28934 time to create 1 rle with old method : 0.03287696838378906 time for calcul the mask position with numpy : 0.03195023536682129 nb_pixel_total : 19553 time to create 1 rle with old method : 0.022713422775268555 time for calcul the mask position with numpy : 0.02951645851135254 nb_pixel_total : 10934 time to create 1 rle with old method : 0.015384912490844727 time for calcul the mask position with numpy : 0.030150175094604492 nb_pixel_total : 12486 time to create 1 rle with old method : 0.013725757598876953 time for calcul the mask position with numpy : 0.030275821685791016 nb_pixel_total : 23530 time to create 1 rle with old method : 0.026033401489257812 create new chi : 2.241220235824585 time to delete rle : 0.010247230529785156 batch 1 Loaded 29 chid ids of type : 3726 Number RLEs to save : 16221 TO DO : save crop sub photo not yet done ! save time : 0.8887560367584229 nb_obj : 31 nb_hashtags : 6 time to prepare the origin masks : 11.96926760673523 time for calcul the mask position with numpy : 0.08539438247680664 nb_pixel_total : 5787047 time to create 1 rle with new method : 0.14492392539978027 time for calcul the mask position with numpy : 0.03290104866027832 nb_pixel_total : 29761 time to create 1 rle with old method : 0.031760215759277344 time for calcul the mask position with numpy : 0.029752254486083984 nb_pixel_total : 65894 time to create 1 rle with old method : 0.07123255729675293 time for calcul the mask position with numpy : 0.029869556427001953 nb_pixel_total : 27327 time to create 1 rle with old method : 0.03023815155029297 time for calcul the mask position with numpy : 0.02921915054321289 nb_pixel_total : 18735 time to create 1 rle with old method : 0.02104926109313965 time for calcul the mask position with numpy : 0.029955625534057617 nb_pixel_total : 8101 time to create 1 rle with old method : 0.00893092155456543 time for calcul the mask position with numpy : 0.029580116271972656 nb_pixel_total : 1268 time to create 1 rle with old method : 0.001500844955444336 time for calcul the mask position with numpy : 0.029349088668823242 nb_pixel_total : 32122 time to create 1 rle with old method : 0.03570961952209473 time for calcul the mask position with numpy : 0.0309140682220459 nb_pixel_total : 123732 time to create 1 rle with old method : 0.1341698169708252 time for calcul the mask position with numpy : 0.029746294021606445 nb_pixel_total : 104577 time to create 1 rle with old method : 0.11531639099121094 time for calcul the mask position with numpy : 0.030694961547851562 nb_pixel_total : 44081 time to create 1 rle with old method : 0.0667579174041748 time for calcul the mask position with numpy : 0.03201699256896973 nb_pixel_total : 662 time to create 1 rle with old method : 0.0008866786956787109 time for calcul the mask position with numpy : 0.029084444046020508 nb_pixel_total : 24595 time to create 1 rle with old method : 0.02663731575012207 time for calcul the mask position with numpy : 0.030912160873413086 nb_pixel_total : 264899 time to create 1 rle with new method : 0.13649344444274902 time for calcul the mask position with numpy : 0.02910590171813965 nb_pixel_total : 32732 time to create 1 rle with old method : 0.035444021224975586 time for calcul the mask position with numpy : 0.029141664505004883 nb_pixel_total : 2121 time to create 1 rle with old method : 0.00261688232421875 time for calcul the mask position with numpy : 0.029241085052490234 nb_pixel_total : 10280 time to create 1 rle with old method : 0.011200189590454102 time for calcul the mask position with numpy : 0.029239654541015625 nb_pixel_total : 3511 time to create 1 rle with old method : 0.004057645797729492 time for calcul the mask position with numpy : 0.029332637786865234 nb_pixel_total : 103821 time to create 1 rle with old method : 0.1116945743560791 time for calcul the mask position with numpy : 0.0288693904876709 nb_pixel_total : 3489 time to create 1 rle with old method : 0.004034996032714844 time for calcul the mask position with numpy : 0.02885150909423828 nb_pixel_total : 57521 time to create 1 rle with old method : 0.06260156631469727 time for calcul the mask position with numpy : 0.028796672821044922 nb_pixel_total : 31080 time to create 1 rle with old method : 0.03405618667602539 time for calcul the mask position with numpy : 0.028220653533935547 nb_pixel_total : 11448 time to create 1 rle with old method : 0.012395620346069336 time for calcul the mask position with numpy : 0.028449296951293945 nb_pixel_total : 33008 time to create 1 rle with old method : 0.038077592849731445 time for calcul the mask position with numpy : 0.028638362884521484 nb_pixel_total : 24808 time to create 1 rle with old method : 0.0270538330078125 time for calcul the mask position with numpy : 0.02859020233154297 nb_pixel_total : 48171 time to create 1 rle with old method : 0.05362057685852051 time for calcul the mask position with numpy : 0.02913355827331543 nb_pixel_total : 35012 time to create 1 rle with old method : 0.038614749908447266 time for calcul the mask position with numpy : 0.030856609344482422 nb_pixel_total : 41040 time to create 1 rle with old method : 0.04305601119995117 time for calcul the mask position with numpy : 0.02906203269958496 nb_pixel_total : 6446 time to create 1 rle with old method : 0.007646799087524414 time for calcul the mask position with numpy : 0.02904224395751953 nb_pixel_total : 20939 time to create 1 rle with old method : 0.02304983139038086 time for calcul the mask position with numpy : 0.0291898250579834 nb_pixel_total : 12353 time to create 1 rle with old method : 0.014152288436889648 time for calcul the mask position with numpy : 0.02904963493347168 nb_pixel_total : 39659 time to create 1 rle with old method : 0.043468475341796875 create new chi : 2.419135808944702 time to delete rle : 0.0021092891693115234 batch 1 Loaded 32 chid ids of type : 3726 Number RLEs to save : 18468 TO DO : save crop sub photo not yet done ! save time : 0.9798474311828613 nb_obj : 10 nb_hashtags : 3 time to prepare the origin masks : 2.997492790222168 time for calcul the mask position with numpy : 0.04961371421813965 nb_pixel_total : 3123875 time to create 1 rle with new method : 0.13332366943359375 time for calcul the mask position with numpy : 0.02030777931213379 nb_pixel_total : 33331 time to create 1 rle with old method : 0.034895896911621094 time for calcul the mask position with numpy : 0.020735740661621094 nb_pixel_total : 138819 time to create 1 rle with old method : 0.1462106704711914 time for calcul the mask position with numpy : 0.022302865982055664 nb_pixel_total : 321025 time to create 1 rle with new method : 0.12126493453979492 time for calcul the mask position with numpy : 0.021840333938598633 nb_pixel_total : 58872 time to create 1 rle with old method : 0.06247973442077637 time for calcul the mask position with numpy : 0.020764589309692383 nb_pixel_total : 244606 time to create 1 rle with new method : 0.1254432201385498 time for calcul the mask position with numpy : 0.01988530158996582 nb_pixel_total : 35390 time to create 1 rle with old method : 0.037200212478637695 time for calcul the mask position with numpy : 0.03013896942138672 nb_pixel_total : 1059855 time to create 1 rle with new method : 0.1245417594909668 time for calcul the mask position with numpy : 0.019893646240234375 nb_pixel_total : 14370 time to create 1 rle with old method : 0.014769315719604492 time for calcul the mask position with numpy : 0.039307355880737305 nb_pixel_total : 1801464 time to create 1 rle with new method : 0.13309240341186523 time for calcul the mask position with numpy : 0.020918607711791992 nb_pixel_total : 218633 time to create 1 rle with new method : 0.13226723670959473 create new chi : 1.417506456375122 time to delete rle : 0.0015947818756103516 batch 1 Loaded 11 chid ids of type : 3726 Number RLEs to save : 19919 TO DO : save crop sub photo not yet done ! save time : 1.2108800411224365 nb_obj : 22 nb_hashtags : 6 time to prepare the origin masks : 5.572723865509033 time for calcul the mask position with numpy : 0.0703573226928711 nb_pixel_total : 5164321 time to create 1 rle with new method : 0.3390352725982666 time for calcul the mask position with numpy : 0.01954507827758789 nb_pixel_total : 8128 time to create 1 rle with old method : 0.008538007736206055 time for calcul the mask position with numpy : 0.01938486099243164 nb_pixel_total : 100945 time to create 1 rle with old method : 0.10322809219360352 time for calcul the mask position with numpy : 0.019961118698120117 nb_pixel_total : 10415 time to create 1 rle with old method : 0.011356353759765625 time for calcul the mask position with numpy : 0.020744800567626953 nb_pixel_total : 84300 time to create 1 rle with old method : 0.0872046947479248 time for calcul the mask position with numpy : 0.022563934326171875 nb_pixel_total : 438364 time to create 1 rle with new method : 0.12541460990905762 time for calcul the mask position with numpy : 0.022608041763305664 nb_pixel_total : 130603 time to create 1 rle with old method : 0.13477087020874023 time for calcul the mask position with numpy : 0.021565914154052734 nb_pixel_total : 192403 time to create 1 rle with new method : 0.13012290000915527 time for calcul the mask position with numpy : 0.01997232437133789 nb_pixel_total : 49491 time to create 1 rle with old method : 0.05318927764892578 time for calcul the mask position with numpy : 0.02077341079711914 nb_pixel_total : 133388 time to create 1 rle with old method : 0.1397714614868164 time for calcul the mask position with numpy : 0.019896507263183594 nb_pixel_total : 3123 time to create 1 rle with old method : 0.003401041030883789 time for calcul the mask position with numpy : 0.0202023983001709 nb_pixel_total : 16870 time to create 1 rle with old method : 0.01857757568359375 time for calcul the mask position with numpy : 0.020491838455200195 nb_pixel_total : 38526 time to create 1 rle with old method : 0.04122638702392578 time for calcul the mask position with numpy : 0.02008056640625 nb_pixel_total : 1583 time to create 1 rle with old method : 0.001817464828491211 time for calcul the mask position with numpy : 0.020073890686035156 nb_pixel_total : 77615 time to create 1 rle with old method : 0.08179092407226562 time for calcul the mask position with numpy : 0.01971602439880371 nb_pixel_total : 18246 time to create 1 rle with old method : 0.019449710845947266 time for calcul the mask position with numpy : 0.01975393295288086 nb_pixel_total : 50829 time to create 1 rle with old method : 0.051287174224853516 time for calcul the mask position with numpy : 0.019466876983642578 nb_pixel_total : 11613 time to create 1 rle with old method : 0.012427806854248047 time for calcul the mask position with numpy : 0.0209200382232666 nb_pixel_total : 151769 time to create 1 rle with new method : 0.12697720527648926 time for calcul the mask position with numpy : 0.020824670791625977 nb_pixel_total : 162946 time to create 1 rle with new method : 0.12570810317993164 time for calcul the mask position with numpy : 0.020570993423461914 nb_pixel_total : 54485 time to create 1 rle with old method : 0.05705618858337402 time for calcul the mask position with numpy : 0.020267486572265625 nb_pixel_total : 45380 time to create 1 rle with old method : 0.0471954345703125 time for calcul the mask position with numpy : 0.019634723663330078 nb_pixel_total : 104897 time to create 1 rle with old method : 0.11115050315856934 create new chi : 2.4259657859802246 time to delete rle : 0.0017838478088378906 batch 1 Loaded 23 chid ids of type : 3726 Number RLEs to save : 21033 TO DO : save crop sub photo not yet done ! save time : 2.0196332931518555 nb_obj : 7 nb_hashtags : 4 time to prepare the origin masks : 2.5852839946746826 time for calcul the mask position with numpy : 0.06298470497131348 nb_pixel_total : 3725343 time to create 1 rle with new method : 0.1442413330078125 time for calcul the mask position with numpy : 0.022169828414916992 nb_pixel_total : 35129 time to create 1 rle with old method : 0.03504610061645508 time for calcul the mask position with numpy : 0.02162027359008789 nb_pixel_total : 8691 time to create 1 rle with old method : 0.009634017944335938 time for calcul the mask position with numpy : 0.02172231674194336 nb_pixel_total : 42083 time to create 1 rle with old method : 0.04210352897644043 time for calcul the mask position with numpy : 0.022600889205932617 nb_pixel_total : 160554 time to create 1 rle with new method : 0.13642001152038574 time for calcul the mask position with numpy : 0.05545473098754883 nb_pixel_total : 3050335 time to create 1 rle with new method : 0.1430037021636963 time for calcul the mask position with numpy : 0.022812366485595703 nb_pixel_total : 24786 time to create 1 rle with old method : 0.02716541290283203 time for calcul the mask position with numpy : 0.021749258041381836 nb_pixel_total : 3319 time to create 1 rle with old method : 0.0035567283630371094 create new chi : 0.8241071701049805 time to delete rle : 0.0012998580932617188 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 13972 TO DO : save crop sub photo not yet done ! save time : 0.8347494602203369 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 4.259435415267944 time for calcul the mask position with numpy : 0.09354925155639648 nb_pixel_total : 6272533 time to create 1 rle with new method : 0.15206480026245117 time for calcul the mask position with numpy : 0.02328777313232422 nb_pixel_total : 182280 time to create 1 rle with new method : 0.13739871978759766 time for calcul the mask position with numpy : 0.022125244140625 nb_pixel_total : 3428 time to create 1 rle with old method : 0.003690004348754883 time for calcul the mask position with numpy : 0.02167510986328125 nb_pixel_total : 2258 time to create 1 rle with old method : 0.0025336742401123047 time for calcul the mask position with numpy : 0.021538734436035156 nb_pixel_total : 7077 time to create 1 rle with old method : 0.0075225830078125 time for calcul the mask position with numpy : 0.023012638092041016 nb_pixel_total : 124632 time to create 1 rle with old method : 0.12412166595458984 time for calcul the mask position with numpy : 0.021489620208740234 nb_pixel_total : 5174 time to create 1 rle with old method : 0.0054738521575927734 time for calcul the mask position with numpy : 0.023142099380493164 nb_pixel_total : 147327 time to create 1 rle with old method : 0.1471717357635498 time for calcul the mask position with numpy : 0.02303767204284668 nb_pixel_total : 67967 time to create 1 rle with old method : 0.06846261024475098 time for calcul the mask position with numpy : 0.024835824966430664 nb_pixel_total : 10936 time to create 1 rle with old method : 0.011797189712524414 time for calcul the mask position with numpy : 0.023152589797973633 nb_pixel_total : 145129 time to create 1 rle with old method : 0.14547038078308105 time for calcul the mask position with numpy : 0.02358555793762207 nb_pixel_total : 79173 time to create 1 rle with old method : 0.08556079864501953 time for calcul the mask position with numpy : 0.023047447204589844 nb_pixel_total : 2326 time to create 1 rle with old method : 0.0026710033416748047 create new chi : 1.2837958335876465 time to delete rle : 0.0009670257568359375 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 8632 TO DO : save crop sub photo not yet done ! save time : 0.4851186275482178 nb_obj : 29 nb_hashtags : 6 time to prepare the origin masks : 13.111595869064331 time for calcul the mask position with numpy : 0.061057329177856445 nb_pixel_total : 3917047 time to create 1 rle with new method : 0.14557719230651855 time for calcul the mask position with numpy : 0.029851913452148438 nb_pixel_total : 35786 time to create 1 rle with old method : 0.03930091857910156 time for calcul the mask position with numpy : 0.03298664093017578 nb_pixel_total : 93831 time to create 1 rle with old method : 0.10168576240539551 time for calcul the mask position with numpy : 0.0299072265625 nb_pixel_total : 73888 time to create 1 rle with old method : 0.0888216495513916 time for calcul the mask position with numpy : 0.03755450248718262 nb_pixel_total : 409999 time to create 1 rle with new method : 0.12943482398986816 time for calcul the mask position with numpy : 0.030213117599487305 nb_pixel_total : 58338 time to create 1 rle with old method : 0.06280255317687988 time for calcul the mask position with numpy : 0.037819623947143555 nb_pixel_total : 1042100 time to create 1 rle with new method : 0.12645673751831055 time for calcul the mask position with numpy : 0.029371261596679688 nb_pixel_total : 143172 time to create 1 rle with old method : 0.18049240112304688 time for calcul the mask position with numpy : 0.029424667358398438 nb_pixel_total : 106086 time to create 1 rle with old method : 0.11494231224060059 time for calcul the mask position with numpy : 0.029008150100708008 nb_pixel_total : 10662 time to create 1 rle with old method : 0.012187719345092773 time for calcul the mask position with numpy : 0.02859640121459961 nb_pixel_total : 5864 time to create 1 rle with old method : 0.0067005157470703125 time for calcul the mask position with numpy : 0.029329299926757812 nb_pixel_total : 103642 time to create 1 rle with old method : 0.11144089698791504 time for calcul the mask position with numpy : 0.028721094131469727 nb_pixel_total : 5849 time to create 1 rle with old method : 0.0066547393798828125 time for calcul the mask position with numpy : 0.02913975715637207 nb_pixel_total : 59362 time to create 1 rle with old method : 0.06483983993530273 time for calcul the mask position with numpy : 0.029434919357299805 nb_pixel_total : 9553 time to create 1 rle with old method : 0.010893583297729492 time for calcul the mask position with numpy : 0.029297590255737305 nb_pixel_total : 116642 time to create 1 rle with old method : 0.13003087043762207 time for calcul the mask position with numpy : 0.030774831771850586 nb_pixel_total : 246889 time to create 1 rle with new method : 0.12273740768432617 time for calcul the mask position with numpy : 0.029641389846801758 nb_pixel_total : 210880 time to create 1 rle with new method : 0.13854074478149414 time for calcul the mask position with numpy : 0.04043698310852051 nb_pixel_total : 12565 time to create 1 rle with old method : 0.014991044998168945 time for calcul the mask position with numpy : 0.029904842376708984 nb_pixel_total : 120 time to create 1 rle with old method : 0.00022029876708984375 time for calcul the mask position with numpy : 0.03291773796081543 nb_pixel_total : 29124 time to create 1 rle with old method : 0.04017186164855957 time for calcul the mask position with numpy : 0.029053211212158203 nb_pixel_total : 2908 time to create 1 rle with old method : 0.003238677978515625 time for calcul the mask position with numpy : 0.02910304069519043 nb_pixel_total : 25458 time to create 1 rle with old method : 0.028104782104492188 time for calcul the mask position with numpy : 0.02993297576904297 nb_pixel_total : 105979 time to create 1 rle with old method : 0.113922119140625 time for calcul the mask position with numpy : 0.029834508895874023 nb_pixel_total : 20381 time to create 1 rle with old method : 0.022259235382080078 time for calcul the mask position with numpy : 0.02939748764038086 nb_pixel_total : 5514 time to create 1 rle with old method : 0.006697416305541992 time for calcul the mask position with numpy : 0.03007650375366211 nb_pixel_total : 88279 time to create 1 rle with old method : 0.09459567070007324 time for calcul the mask position with numpy : 0.029709577560424805 nb_pixel_total : 29570 time to create 1 rle with old method : 0.03260970115661621 time for calcul the mask position with numpy : 0.029822349548339844 nb_pixel_total : 73826 time to create 1 rle with old method : 0.07903409004211426 time for calcul the mask position with numpy : 0.029195785522460938 nb_pixel_total : 6926 time to create 1 rle with old method : 0.007611513137817383 create new chi : 3.0585763454437256 time to delete rle : 0.002284526824951172 batch 1 Loaded 30 chid ids of type : 3726 Number RLEs to save : 26611 TO DO : save crop sub photo not yet done ! save time : 1.4641897678375244 nb_obj : 15 nb_hashtags : 6 time to prepare the origin masks : 4.226355075836182 time for calcul the mask position with numpy : 0.20627641677856445 nb_pixel_total : 6606626 time to create 1 rle with new method : 0.3218858242034912 time for calcul the mask position with numpy : 0.02294182777404785 nb_pixel_total : 63601 time to create 1 rle with old method : 0.06549334526062012 time for calcul the mask position with numpy : 0.022548675537109375 nb_pixel_total : 17694 time to create 1 rle with old method : 0.01856088638305664 time for calcul the mask position with numpy : 0.02191925048828125 nb_pixel_total : 2389 time to create 1 rle with old method : 0.0025818347930908203 time for calcul the mask position with numpy : 0.022063493728637695 nb_pixel_total : 18983 time to create 1 rle with old method : 0.019886016845703125 time for calcul the mask position with numpy : 0.0219113826751709 nb_pixel_total : 8897 time to create 1 rle with old method : 0.009261369705200195 time for calcul the mask position with numpy : 0.021851778030395508 nb_pixel_total : 29061 time to create 1 rle with old method : 0.0301816463470459 time for calcul the mask position with numpy : 0.02335381507873535 nb_pixel_total : 13377 time to create 1 rle with old method : 0.014549970626831055 time for calcul the mask position with numpy : 0.023116350173950195 nb_pixel_total : 147138 time to create 1 rle with old method : 0.15516328811645508 time for calcul the mask position with numpy : 0.022422075271606445 nb_pixel_total : 7149 time to create 1 rle with old method : 0.0075037479400634766 time for calcul the mask position with numpy : 0.022027015686035156 nb_pixel_total : 82953 time to create 1 rle with old method : 0.0822606086730957 time for calcul the mask position with numpy : 0.021649837493896484 nb_pixel_total : 6282 time to create 1 rle with old method : 0.006796121597290039 time for calcul the mask position with numpy : 0.02186727523803711 nb_pixel_total : 4647 time to create 1 rle with old method : 0.0049364566802978516 time for calcul the mask position with numpy : 0.022084712982177734 nb_pixel_total : 13453 time to create 1 rle with old method : 0.01367497444152832 time for calcul the mask position with numpy : 0.022745847702026367 nb_pixel_total : 10704 time to create 1 rle with old method : 0.011471748352050781 time for calcul the mask position with numpy : 0.022776126861572266 nb_pixel_total : 17286 time to create 1 rle with old method : 0.018161296844482422 create new chi : 1.354954481124878 time to delete rle : 0.0008440017700195312 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 8706 TO DO : save crop sub photo not yet done ! save time : 0.5027885437011719 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 3.085725784301758 time for calcul the mask position with numpy : 0.09910011291503906 nb_pixel_total : 6435922 time to create 1 rle with new method : 0.17262053489685059 time for calcul the mask position with numpy : 0.020407676696777344 nb_pixel_total : 9995 time to create 1 rle with old method : 0.011126995086669922 time for calcul the mask position with numpy : 0.020446300506591797 nb_pixel_total : 56167 time to create 1 rle with old method : 0.05971550941467285 time for calcul the mask position with numpy : 0.01956653594970703 nb_pixel_total : 6095 time to create 1 rle with old method : 0.006724834442138672 time for calcul the mask position with numpy : 0.0198211669921875 nb_pixel_total : 1239 time to create 1 rle with old method : 0.001359701156616211 time for calcul the mask position with numpy : 0.01973271369934082 nb_pixel_total : 28997 time to create 1 rle with old method : 0.02956676483154297 time for calcul the mask position with numpy : 0.021879911422729492 nb_pixel_total : 61126 time to create 1 rle with old method : 0.0622403621673584 time for calcul the mask position with numpy : 0.020586729049682617 nb_pixel_total : 26745 time to create 1 rle with old method : 0.026641845703125 time for calcul the mask position with numpy : 0.019838571548461914 nb_pixel_total : 12893 time to create 1 rle with old method : 0.014106988906860352 time for calcul the mask position with numpy : 0.02063131332397461 nb_pixel_total : 5095 time to create 1 rle with old method : 0.0054852962493896484 time for calcul the mask position with numpy : 0.020647525787353516 nb_pixel_total : 13803 time to create 1 rle with old method : 0.01510477066040039 time for calcul the mask position with numpy : 0.022513628005981445 nb_pixel_total : 307757 time to create 1 rle with new method : 0.1353013515472412 time for calcul the mask position with numpy : 0.025119781494140625 nb_pixel_total : 5088 time to create 1 rle with old method : 0.005952119827270508 time for calcul the mask position with numpy : 0.02103400230407715 nb_pixel_total : 76776 time to create 1 rle with old method : 0.07898736000061035 time for calcul the mask position with numpy : 0.019918441772460938 nb_pixel_total : 2542 time to create 1 rle with old method : 0.0026464462280273438 create new chi : 1.0497140884399414 time to delete rle : 0.0008058547973632812 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 7971 TO DO : save crop sub photo not yet done ! save time : 0.46332478523254395 nb_obj : 4 nb_hashtags : 2 time to prepare the origin masks : 1.0673561096191406 time for calcul the mask position with numpy : 0.045151472091674805 nb_pixel_total : 2896461 time to create 1 rle with new method : 0.13379573822021484 time for calcul the mask position with numpy : 0.020792722702026367 nb_pixel_total : 27697 time to create 1 rle with old method : 0.030137062072753906 time for calcul the mask position with numpy : 0.02261972427368164 nb_pixel_total : 363897 time to create 1 rle with new method : 0.1247859001159668 time for calcul the mask position with numpy : 0.022198200225830078 nb_pixel_total : 381864 time to create 1 rle with new method : 0.13123440742492676 time for calcul the mask position with numpy : 0.04732847213745117 nb_pixel_total : 3380321 time to create 1 rle with new method : 0.13131093978881836 create new chi : 0.7575538158416748 time to delete rle : 0.0014874935150146484 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 12584 TO DO : save crop sub photo not yet done ! save time : 0.6986072063446045 map_output_result : {1334287114: (0.0, 'Should be the crop_list due to order', 0.0), 1334287106: (0.0, 'Should be the crop_list due to order', 0.0), 1334287082: (0.0, 'Should be the crop_list due to order', 0.0), 1334287033: (0.0, 'Should be the crop_list due to order', 0.0), 1334286966: (0.0, 'Should be the crop_list due to order', 0.0), 1334285826: (0.0, 'Should be the crop_list due to order', 0.0), 1334285724: (0.0, 'Should be the crop_list due to order', 0.0), 1334285397: (0.0, 'Should be the crop_list due to order', 0.0), 1334285394: (0.0, 'Should be the crop_list due to order', 0.0), 1334285376: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1334287114, 1334287106, 1334287082, 1334287033, 1334286966, 1334285826, 1334285724, 1334285397, 1334285394, 1334285376] Looping around the photos to save general results len do output : 10 /1334287114.Didn't retrieve data . /1334287106.Didn't retrieve data . /1334287082.Didn't retrieve data . /1334287033.Didn't retrieve data . /1334286966.Didn't retrieve data . /1334285826.Didn't retrieve data . /1334285724.Didn't retrieve data . /1334285397.Didn't retrieve data . /1334285394.Didn't retrieve data . /1334285376.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287114', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287106', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287082', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287033', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334286966', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285826', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285724', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285397', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285394', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285376', None, None, None, None, None, '2542528') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.012472867965698242 save_final save missing photos in datou_result : time spend for datou_step_exec : 86.5411307811737 time spend to save output : 0.013092041015625 total time spend for step 5 : 86.55422282218933 step6:crop_condition Mon Feb 3 22:37:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 182 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738618679_1868179 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.8250739574432373 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738618689_1868179 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.291961669921875 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738618694_1868179 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.787339210510254 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738618697_1868179 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9512097835540771 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738618700_1868179 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.0676867961883545 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738618705_1868179 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.379662036895752 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1334287114, 1334287106, 1334287082, 1334287033, 1334286966, 1334285826, 1334285724, 1334285397, 1334285394, 1334285376] Looping around the photos to save general results len do output : 37 /1334349819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349854Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349857Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349858Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349861Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349864Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349867Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349868Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349874Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349912Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349917Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1334349918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287114', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287106', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287082', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287033', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334286966', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285826', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285724', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285397', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285394', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285376', None, None, None, None, None, '2542528') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 121 time used for this insertion : 0.032996416091918945 save_final save missing photos in datou_result : time spend for datou_step_exec : 33.148818016052246 time spend to save output : 0.03411412239074707 total time spend for step 6 : 33.18293213844299 step7:ventilate_hashtags_in_portfolio Mon Feb 3 22:38:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20193384 get user id for portfolio 20193384 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20193384 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','papier','autre_refus','plastique','mal_croppe','environnement','metal','flou','Teint_Dans_La_Masse','cartonnette','kraft','Carton_gris')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20193384 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','papier','autre_refus','plastique','mal_croppe','environnement','metal','flou','Teint_Dans_La_Masse','cartonnette','kraft','Carton_gris')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20193384 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_brun','papier','autre_refus','plastique','mal_croppe','environnement','metal','flou','Teint_Dans_La_Masse','cartonnette','kraft','Carton_gris')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20193562,20193563,20193564,20193565,20193566,20193567,20193568,20193569,20193570,20193571,20193572,20193573?tags=Carton_brun,papier,autre_refus,plastique,mal_croppe,environnement,metal,flou,Teint_Dans_La_Masse,cartonnette,kraft,Carton_gris Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1334287114, 1334287106, 1334287082, 1334287033, 1334286966, 1334285826, 1334285724, 1334285397, 1334285394, 1334285376] Looping around the photos to save general results len do output : 1 /20193384. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287114', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287106', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287082', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287033', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334286966', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285826', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285724', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285397', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285394', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285376', None, None, None, None, None, '2542528') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.013710737228393555 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.6891498565673828 time spend to save output : 0.014100313186645508 total time spend for step 7 : 1.7032501697540283 step8:final Mon Feb 3 22:38:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1334287114: ('0.096394624967941',), 1334287106: ('0.096394624967941',), 1334287082: ('0.096394624967941',), 1334287033: ('0.096394624967941',), 1334286966: ('0.096394624967941',), 1334285826: ('0.096394624967941',), 1334285724: ('0.096394624967941',), 1334285397: ('0.096394624967941',), 1334285394: ('0.096394624967941',), 1334285376: ('0.096394624967941',)} new output for save of step final : {1334287114: ('0.096394624967941',), 1334287106: ('0.096394624967941',), 1334287082: ('0.096394624967941',), 1334287033: ('0.096394624967941',), 1334286966: ('0.096394624967941',), 1334285826: ('0.096394624967941',), 1334285724: ('0.096394624967941',), 1334285397: ('0.096394624967941',), 1334285394: ('0.096394624967941',), 1334285376: ('0.096394624967941',)} [1334287114, 1334287106, 1334287082, 1334287033, 1334286966, 1334285826, 1334285724, 1334285397, 1334285394, 1334285376] Looping around the photos to save general results len do output : 10 /1334287114.Didn't retrieve data . /1334287106.Didn't retrieve data . /1334287082.Didn't retrieve data . /1334287033.Didn't retrieve data . /1334286966.Didn't retrieve data . /1334285826.Didn't retrieve data . /1334285724.Didn't retrieve data . /1334285397.Didn't retrieve data . /1334285394.Didn't retrieve data . /1334285376.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287114', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287106', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287082', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287033', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334286966', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285826', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285724', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285397', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285394', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285376', None, None, None, None, None, '2542528') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013721227645874023 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.09380602836608887 time spend to save output : 0.014354228973388672 total time spend for step 8 : 0.10816025733947754 step9:velours_tree Mon Feb 3 22:38:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.05556631088256836 time spend to save output : 3.528594970703125e-05 total time spend for step 9 : 0.05560159683227539 step10:send_mail_cod Mon Feb 3 22:38:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20193384_03-02-2025_22_38_29.pdf 20193564 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201935641738618709 20193565 change filename to text .change filename to text .change filename to text .change filename to text .imagette201935651738618710 20193566 imagette201935661738618710 20193568 imagette201935681738618710 20193569 imagette201935691738618710 20193571 change filename to text .change filename to text .change filename to text .imagette201935711738618710 20193572 imagette201935721738618710 20193562 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201935621738618710 20193570 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201935701738618710 20193573 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette201935731738618711 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20193384 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20193562,20193563,20193564,20193565,20193566,20193567,20193568,20193569,20193570,20193571,20193572,20193573?tags=Carton_brun,papier,autre_refus,plastique,mal_croppe,environnement,metal,flou,Teint_Dans_La_Masse,cartonnette,kraft,Carton_gris your option no_mail is active, we will not send the real mail to your client args[1334287114] : ('0.096394624967941',) no score found for photo 1334287114 We are sending mail with results at report@fotonower.com args[1334287106] : ('0.096394624967941',) no score found for photo 1334287106 We are sending mail with results at report@fotonower.com args[1334287082] : ('0.096394624967941',) no score found for photo 1334287082 We are sending mail with results at report@fotonower.com args[1334287033] : ('0.096394624967941',) no score found for photo 1334287033 We are sending mail with results at report@fotonower.com args[1334286966] : ('0.096394624967941',) no score found for photo 1334286966 We are sending mail with results at report@fotonower.com args[1334285826] : ('0.096394624967941',) no score found for photo 1334285826 We are sending mail with results at report@fotonower.com args[1334285724] : ('0.096394624967941',) no score found for photo 1334285724 We are sending mail with results at report@fotonower.com args[1334285397] : ('0.096394624967941',) no score found for photo 1334285397 We are sending mail with results at report@fotonower.com args[1334285394] : ('0.096394624967941',) no score found for photo 1334285394 We are sending mail with results at report@fotonower.com args[1334285376] : ('0.096394624967941',) no score found for photo 1334285376 We are sending mail with results at report@fotonower.com refus_total : 0.096394624967941 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20193384 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1334285338,1334285397,1334285724,1334285826,1334287033,1334287106,1334287114,1334285295,1334285318,1334285376,1334285394,1334286966,1334287082) Found this number of photos: 13 begin to download photo : 1334285338 begin to download photo : 1334285826 begin to download photo : 1334287114 begin to download photo : 1334285376 begin to download photo : 1334287082 download finish for photo 1334287114 begin to download photo : 1334285295 download finish for photo 1334285338 begin to download photo : 1334285397 download finish for photo 1334287082 download finish for photo 1334285826 begin to download photo : 1334287033 download finish for photo 1334287033 begin to download photo : 1334287106 download finish for photo 1334285295 begin to download photo : 1334285318 download finish for photo 1334285397 begin to download photo : 1334285724 download finish for photo 1334285318 download finish for photo 1334287106 download finish for photo 1334285376 begin to download photo : 1334285394 download finish for photo 1334285724 download finish for photo 1334285394 begin to download photo : 1334286966 download finish for photo 1334286966 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20193384_03-02-2025_22_38_29.pdf results_Auto_P20193384_03-02-2025_22_38_29.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20193384_03-02-2025_22_38_29.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20193384','results_Auto_P20193384_03-02-2025_22_38_29.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20193384_03-02-2025_22_38_29.pdf','pdf','','0.7','0.096394624967941') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1334287114, 1334287106, 1334287082, 1334287033, 1334286966, 1334285826, 1334285724, 1334285397, 1334285394, 1334285376] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287114', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287106', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287082', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287033', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334286966', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285826', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285724', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285397', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285394', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285376', None, None, None, None, None, '2542528') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013383865356445312 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.566922664642334 time spend to save output : 0.01370692253112793 total time spend for step 10 : 5.580629587173462 step11:split_time_score Mon Feb 3 22:38:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('18', 13),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 03022025 20193384 Nombre de photos uploadées : 13 / 23040 (0%) 03022025 20193384 Nombre de photos taguées (types de déchets): 0 / 13 (0%) 03022025 20193384 Nombre de photos taguées (volume) : 0 / 13 (0%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0006589889526367188 elapsed_time : insert_dashboard_record_day_entry 0.025291919708251953 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193381 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193382 order by id desc limit 1 Qualite : 0.11132732071472617 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187042_03-02-2025_19_58_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187042 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187042 AND mptpi.`type`=4200 To do Qualite : 0.04468680087730703 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187043_03-02-2025_21_26_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187043 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187043 AND mptpi.`type`=4207 To do Qualite : 0.05126391356360632 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187044_03-02-2025_21_54_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187044 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187044 AND mptpi.`type`=4207 To do Qualite : 0.055936445147032034 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187063_03-02-2025_22_35_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187063 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187063 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187070 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187083 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187084 order by id desc limit 1 Qualite : 0.13719855596908986 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187101_03-02-2025_18_32_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187101 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187101 AND mptpi.`type`=4200 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187102 order by id desc limit 1 Qualite : 0.02707472932635323 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187103_03-02-2025_20_37_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187103 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187103 AND mptpi.`type`=4207 To do Qualite : 0.036651538371265856 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187104_03-02-2025_20_10_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187104 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187104 AND mptpi.`type`=4207 To do Qualite : 0.043481678214483135 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187105_03-02-2025_22_25_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187105 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187105 AND mptpi.`type`=4207 To do Qualite : 0.0383331876813021 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187106_03-02-2025_19_53_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187106 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187106 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187107 order by id desc limit 1 Qualite : 0.06485345527187093 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187108_03-02-2025_19_16_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187108 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187108 AND mptpi.`type`=4207 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187109_03-02-2025_20_07_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187109 order by id desc limit 1 Qualite : 0.220957984151152 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187110_03-02-2025_18_29_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187110 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187110 AND mptpi.`type`=3594 To do Qualite : 0.21167360646910285 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187111_03-02-2025_19_28_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187111 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187111 AND mptpi.`type`=3594 To do Qualite : 0.2511494694880024 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187112_03-02-2025_20_30_42.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187112 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187112 AND mptpi.`type`=4200 To do Qualite : 0.07815820189177887 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187113_03-02-2025_18_55_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187113 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187113 AND mptpi.`type`=3327 To do Qualite : 0.05582247809496911 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187114_03-02-2025_20_24_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187114 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187114 AND mptpi.`type`=4207 To do Qualite : 0.047334585345209 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187115_03-02-2025_18_52_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187115 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187115 AND mptpi.`type`=4207 To do Qualite : 0.08582553244849807 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187116_03-02-2025_18_51_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187116 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187116 AND mptpi.`type`=4207 To do Qualite : 0.04381375648735101 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187117_03-02-2025_18_37_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187117 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187117 AND mptpi.`type`=4207 To do Qualite : 0.20702335610615327 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187118_03-02-2025_21_49_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187118 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187118 AND mptpi.`type`=3594 To do Qualite : 0.16090710157403149 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20187119_03-02-2025_18_19_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187119 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20187119 AND mptpi.`type`=4200 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20187120 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193383 order by id desc limit 1 Qualite : 0.096394624967941 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20193384_03-02-2025_22_38_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193384 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20193384 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193385 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193386 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193387 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193388 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193389 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193390 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193391 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20193392 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'03022025': {'nb_upload': 13, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1334287114, 1334287106, 1334287082, 1334287033, 1334286966, 1334285826, 1334285724, 1334285397, 1334285394, 1334285376] Looping around the photos to save general results len do output : 1 /20193384Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287114', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287106', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287082', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334287033', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334286966', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285826', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285724', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285397', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285394', None, None, None, None, None, '2542528') ('3459', None, None, None, None, None, None, None, '2542528') ('3459', '20193384', '1334285376', None, None, None, None, None, '2542528') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014114618301391602 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.738083839416504 time spend to save output : 0.014397144317626953 total time spend for step 11 : 14.75248098373413 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 190.09user 45.78system 4:25.05elapsed 88%CPU (0avgtext+0avgdata 4787436maxresident)k 2409392inputs+131256outputs (8049major+8637731minor)pagefaults 0swaps