python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 826376 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 19>10 please execute split_portfolio.py -i 20384176 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2569872'] with mtr_portfolio_ids : ['20384176'] and first list_photo_ids : [] new path : /proc/826376/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 9 ; length of list_pids : 9 ; length of list_args : 9 time to download the photos : 1.4855070114135742 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Mon Feb 10 03:04:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10111 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-10 03:04:33.743659: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-10 03:04:33.775173: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-10 03:04:33.777010: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fae4c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-10 03:04:33.777036: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-10 03:04:33.781460: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-10 03:04:34.078907: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x322c43b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-10 03:04:34.078966: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-10 03:04:34.080077: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 03:04:34.081522: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 03:04:34.092606: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 03:04:34.111772: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 03:04:34.115401: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 03:04:34.149129: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 03:04:34.154467: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 03:04:34.213486: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 03:04:34.215343: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 03:04:34.215832: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 03:04:34.217202: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-10 03:04:34.217219: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-10 03:04:34.217228: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-10 03:04:34.218989: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9362 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-10 03:04:34.647184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 03:04:34.647323: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 03:04:34.647340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 03:04:34.647356: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 03:04:34.647370: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 03:04:34.647383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 03:04:34.647397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 03:04:34.647412: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 03:04:34.648571: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 03:04:34.649780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-10 03:04:34.649817: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-10 03:04:34.649835: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 03:04:34.649851: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-10 03:04:34.649867: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-10 03:04:34.649883: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-10 03:04:34.649898: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-10 03:04:34.649915: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-10 03:04:34.651121: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-10 03:04:34.651159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-10 03:04:34.651168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-10 03:04:34.651175: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-10 03:04:34.652369: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9362 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-10 03:04:45.542819: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-10 03:04:45.832094: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 32 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 18 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 16 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 12 Detection mask done ! Trying to reset tf kernel 826750 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5043 tf kernel not reseted sub process len(results) : 9 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 9 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10332 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0012290477752685547 nb_pixel_total : 36685 time to create 1 rle with old method : 0.04233264923095703 length of segment : 304 time for calcul the mask position with numpy : 0.0005948543548583984 nb_pixel_total : 25741 time to create 1 rle with old method : 0.03618216514587402 length of segment : 458 time for calcul the mask position with numpy : 0.002983570098876953 nb_pixel_total : 115530 time to create 1 rle with old method : 0.13062310218811035 length of segment : 519 time for calcul the mask position with numpy : 0.0003693103790283203 nb_pixel_total : 16025 time to create 1 rle with old method : 0.018625259399414062 length of segment : 238 time for calcul the mask position with numpy : 0.0040798187255859375 nb_pixel_total : 156961 time to create 1 rle with new method : 0.014384984970092773 length of segment : 625 time for calcul the mask position with numpy : 0.003433704376220703 nb_pixel_total : 179191 time to create 1 rle with new method : 0.016101598739624023 length of segment : 861 time for calcul the mask position with numpy : 0.0009207725524902344 nb_pixel_total : 46925 time to create 1 rle with old method : 0.05775165557861328 length of segment : 544 time for calcul the mask position with numpy : 0.0008707046508789062 nb_pixel_total : 36016 time to create 1 rle with old method : 0.0411221981048584 length of segment : 309 time for calcul the mask position with numpy : 0.0026307106018066406 nb_pixel_total : 62940 time to create 1 rle with old method : 0.07255434989929199 length of segment : 459 time for calcul the mask position with numpy : 0.0016863346099853516 nb_pixel_total : 91200 time to create 1 rle with old method : 0.10013604164123535 length of segment : 525 time for calcul the mask position with numpy : 0.00024580955505371094 nb_pixel_total : 7035 time to create 1 rle with old method : 0.00856781005859375 length of segment : 81 time for calcul the mask position with numpy : 0.010419845581054688 nb_pixel_total : 534701 time to create 1 rle with new method : 0.0279843807220459 length of segment : 1365 time for calcul the mask position with numpy : 0.0015327930450439453 nb_pixel_total : 58505 time to create 1 rle with old method : 0.06567859649658203 length of segment : 376 time for calcul the mask position with numpy : 0.00010156631469726562 nb_pixel_total : 3135 time to create 1 rle with old method : 0.004130840301513672 length of segment : 58 time for calcul the mask position with numpy : 0.00037360191345214844 nb_pixel_total : 13872 time to create 1 rle with old method : 0.017081260681152344 length of segment : 135 time for calcul the mask position with numpy : 0.0007240772247314453 nb_pixel_total : 46580 time to create 1 rle with old method : 0.05404543876647949 length of segment : 264 time for calcul the mask position with numpy : 0.0006773471832275391 nb_pixel_total : 48075 time to create 1 rle with old method : 0.05614876747131348 length of segment : 247 time for calcul the mask position with numpy : 0.002251148223876953 nb_pixel_total : 147435 time to create 1 rle with old method : 0.19355201721191406 length of segment : 443 time for calcul the mask position with numpy : 0.0025582313537597656 nb_pixel_total : 89769 time to create 1 rle with old method : 0.13650202751159668 length of segment : 507 time for calcul the mask position with numpy : 0.0020394325256347656 nb_pixel_total : 83951 time to create 1 rle with old method : 0.12781667709350586 length of segment : 512 time for calcul the mask position with numpy : 0.0012764930725097656 nb_pixel_total : 48460 time to create 1 rle with old method : 0.06143665313720703 length of segment : 349 time for calcul the mask position with numpy : 0.0008838176727294922 nb_pixel_total : 41648 time to create 1 rle with old method : 0.04900169372558594 length of segment : 215 time for calcul the mask position with numpy : 0.00014853477478027344 nb_pixel_total : 2096 time to create 1 rle with old method : 0.0028829574584960938 length of segment : 52 time for calcul the mask position with numpy : 0.0018930435180664062 nb_pixel_total : 63840 time to create 1 rle with old method : 0.07650089263916016 length of segment : 531 time for calcul the mask position with numpy : 0.003815174102783203 nb_pixel_total : 171718 time to create 1 rle with new method : 0.006042957305908203 length of segment : 265 time for calcul the mask position with numpy : 0.0006833076477050781 nb_pixel_total : 36561 time to create 1 rle with old method : 0.04720020294189453 length of segment : 170 time for calcul the mask position with numpy : 0.0004703998565673828 nb_pixel_total : 22376 time to create 1 rle with old method : 0.026692628860473633 length of segment : 137 time for calcul the mask position with numpy : 0.0010950565338134766 nb_pixel_total : 54245 time to create 1 rle with old method : 0.06548428535461426 length of segment : 371 time for calcul the mask position with numpy : 0.0037908554077148438 nb_pixel_total : 87625 time to create 1 rle with old method : 0.10849857330322266 length of segment : 453 time for calcul the mask position with numpy : 0.0003275871276855469 nb_pixel_total : 3635 time to create 1 rle with old method : 0.0049953460693359375 length of segment : 78 time for calcul the mask position with numpy : 0.0017855167388916016 nb_pixel_total : 43600 time to create 1 rle with old method : 0.052756547927856445 length of segment : 200 time for calcul the mask position with numpy : 0.00029015541076660156 nb_pixel_total : 6151 time to create 1 rle with old method : 0.0074253082275390625 length of segment : 106 time for calcul the mask position with numpy : 0.002110004425048828 nb_pixel_total : 65857 time to create 1 rle with old method : 0.07804226875305176 length of segment : 429 time for calcul the mask position with numpy : 0.00030803680419921875 nb_pixel_total : 7751 time to create 1 rle with old method : 0.009321451187133789 length of segment : 119 time for calcul the mask position with numpy : 0.00395965576171875 nb_pixel_total : 139323 time to create 1 rle with old method : 0.15337228775024414 length of segment : 396 time for calcul the mask position with numpy : 0.00017595291137695312 nb_pixel_total : 1592 time to create 1 rle with old method : 0.0019459724426269531 length of segment : 59 time for calcul the mask position with numpy : 0.0013136863708496094 nb_pixel_total : 25434 time to create 1 rle with old method : 0.02974247932434082 length of segment : 362 time for calcul the mask position with numpy : 0.002489805221557617 nb_pixel_total : 73961 time to create 1 rle with old method : 0.10016798973083496 length of segment : 524 time for calcul the mask position with numpy : 0.005929470062255859 nb_pixel_total : 190291 time to create 1 rle with new method : 0.011761188507080078 length of segment : 525 time for calcul the mask position with numpy : 8.7738037109375e-05 nb_pixel_total : 2140 time to create 1 rle with old method : 0.0027179718017578125 length of segment : 41 time for calcul the mask position with numpy : 0.003850698471069336 nb_pixel_total : 150210 time to create 1 rle with new method : 0.00641632080078125 length of segment : 549 time for calcul the mask position with numpy : 0.003255128860473633 nb_pixel_total : 196963 time to create 1 rle with new method : 0.008602619171142578 length of segment : 453 time for calcul the mask position with numpy : 0.0006897449493408203 nb_pixel_total : 19054 time to create 1 rle with old method : 0.022671937942504883 length of segment : 207 time for calcul the mask position with numpy : 0.003322124481201172 nb_pixel_total : 70317 time to create 1 rle with old method : 0.1036381721496582 length of segment : 286 time for calcul the mask position with numpy : 0.004084348678588867 nb_pixel_total : 190440 time to create 1 rle with new method : 0.008312463760375977 length of segment : 524 time for calcul the mask position with numpy : 0.0004096031188964844 nb_pixel_total : 6453 time to create 1 rle with old method : 0.009259223937988281 length of segment : 100 time for calcul the mask position with numpy : 0.0009918212890625 nb_pixel_total : 42167 time to create 1 rle with old method : 0.048380136489868164 length of segment : 276 time for calcul the mask position with numpy : 0.00021409988403320312 nb_pixel_total : 7999 time to create 1 rle with old method : 0.009949684143066406 length of segment : 72 time for calcul the mask position with numpy : 0.0007710456848144531 nb_pixel_total : 33738 time to create 1 rle with old method : 0.03892874717712402 length of segment : 199 time for calcul the mask position with numpy : 0.0023534297943115234 nb_pixel_total : 82608 time to create 1 rle with old method : 0.09493184089660645 length of segment : 569 time for calcul the mask position with numpy : 0.006957054138183594 nb_pixel_total : 324381 time to create 1 rle with new method : 0.015548229217529297 length of segment : 417 time for calcul the mask position with numpy : 0.0007424354553222656 nb_pixel_total : 39241 time to create 1 rle with old method : 0.045642852783203125 length of segment : 209 time for calcul the mask position with numpy : 0.003760814666748047 nb_pixel_total : 66505 time to create 1 rle with old method : 0.07802700996398926 length of segment : 408 time for calcul the mask position with numpy : 0.002153635025024414 nb_pixel_total : 75770 time to create 1 rle with old method : 0.1083528995513916 length of segment : 495 time for calcul the mask position with numpy : 0.0015347003936767578 nb_pixel_total : 23412 time to create 1 rle with old method : 0.027228116989135742 length of segment : 187 time for calcul the mask position with numpy : 0.008388996124267578 nb_pixel_total : 328645 time to create 1 rle with new method : 0.016951560974121094 length of segment : 498 time for calcul the mask position with numpy : 0.0016570091247558594 nb_pixel_total : 42276 time to create 1 rle with old method : 0.05558896064758301 length of segment : 276 time for calcul the mask position with numpy : 0.0006842613220214844 nb_pixel_total : 12949 time to create 1 rle with old method : 0.016891002655029297 length of segment : 181 time for calcul the mask position with numpy : 0.0035965442657470703 nb_pixel_total : 112207 time to create 1 rle with old method : 0.1334846019744873 length of segment : 450 time for calcul the mask position with numpy : 0.0017321109771728516 nb_pixel_total : 78326 time to create 1 rle with old method : 0.08762645721435547 length of segment : 510 time for calcul the mask position with numpy : 0.001991748809814453 nb_pixel_total : 86984 time to create 1 rle with old method : 0.09781861305236816 length of segment : 352 time for calcul the mask position with numpy : 0.0013515949249267578 nb_pixel_total : 44279 time to create 1 rle with old method : 0.05050086975097656 length of segment : 347 time for calcul the mask position with numpy : 0.009022712707519531 nb_pixel_total : 347115 time to create 1 rle with new method : 0.03752732276916504 length of segment : 854 time for calcul the mask position with numpy : 0.0002830028533935547 nb_pixel_total : 8143 time to create 1 rle with old method : 0.00995016098022461 length of segment : 116 time for calcul the mask position with numpy : 0.0001049041748046875 nb_pixel_total : 2053 time to create 1 rle with old method : 0.002455472946166992 length of segment : 37 time for calcul the mask position with numpy : 0.00022912025451660156 nb_pixel_total : 6366 time to create 1 rle with old method : 0.007753849029541016 length of segment : 84 time for calcul the mask position with numpy : 0.005376577377319336 nb_pixel_total : 347549 time to create 1 rle with new method : 0.26581621170043945 length of segment : 938 time for calcul the mask position with numpy : 0.002065420150756836 nb_pixel_total : 84640 time to create 1 rle with old method : 0.09714579582214355 length of segment : 431 time for calcul the mask position with numpy : 0.00011849403381347656 nb_pixel_total : 2539 time to create 1 rle with old method : 0.003165721893310547 length of segment : 44 time for calcul the mask position with numpy : 0.0003371238708496094 nb_pixel_total : 19922 time to create 1 rle with old method : 0.02306056022644043 length of segment : 153 time for calcul the mask position with numpy : 0.0009450912475585938 nb_pixel_total : 28613 time to create 1 rle with old method : 0.03278231620788574 length of segment : 257 time for calcul the mask position with numpy : 0.00018978118896484375 nb_pixel_total : 2809 time to create 1 rle with old method : 0.003383159637451172 length of segment : 87 time for calcul the mask position with numpy : 0.0006773471832275391 nb_pixel_total : 33280 time to create 1 rle with old method : 0.03848862648010254 length of segment : 300 time for calcul the mask position with numpy : 0.0005071163177490234 nb_pixel_total : 24884 time to create 1 rle with old method : 0.02932143211364746 length of segment : 222 time for calcul the mask position with numpy : 0.0006313323974609375 nb_pixel_total : 27902 time to create 1 rle with old method : 0.03235268592834473 length of segment : 192 time for calcul the mask position with numpy : 0.00041675567626953125 nb_pixel_total : 17198 time to create 1 rle with old method : 0.020668745040893555 length of segment : 119 time for calcul the mask position with numpy : 0.000553131103515625 nb_pixel_total : 18605 time to create 1 rle with old method : 0.021938562393188477 length of segment : 201 time for calcul the mask position with numpy : 0.0001971721649169922 nb_pixel_total : 3318 time to create 1 rle with old method : 0.004043102264404297 length of segment : 94 time for calcul the mask position with numpy : 0.0029909610748291016 nb_pixel_total : 138872 time to create 1 rle with old method : 0.15286612510681152 length of segment : 606 time for calcul the mask position with numpy : 0.00018477439880371094 nb_pixel_total : 7301 time to create 1 rle with old method : 0.008968591690063477 length of segment : 74 time for calcul the mask position with numpy : 0.0003337860107421875 nb_pixel_total : 12053 time to create 1 rle with old method : 0.013957500457763672 length of segment : 151 time for calcul the mask position with numpy : 0.0010945796966552734 nb_pixel_total : 41401 time to create 1 rle with old method : 0.04783344268798828 length of segment : 226 time for calcul the mask position with numpy : 0.00011944770812988281 nb_pixel_total : 2493 time to create 1 rle with old method : 0.003082752227783203 length of segment : 92 time for calcul the mask position with numpy : 0.000316619873046875 nb_pixel_total : 11090 time to create 1 rle with old method : 0.013640642166137695 length of segment : 154 time for calcul the mask position with numpy : 0.0009009838104248047 nb_pixel_total : 33357 time to create 1 rle with old method : 0.037819862365722656 length of segment : 146 time for calcul the mask position with numpy : 0.00036907196044921875 nb_pixel_total : 5629 time to create 1 rle with old method : 0.006827592849731445 length of segment : 125 time for calcul the mask position with numpy : 0.0002586841583251953 nb_pixel_total : 5788 time to create 1 rle with old method : 0.0070192813873291016 length of segment : 81 time for calcul the mask position with numpy : 0.00026154518127441406 nb_pixel_total : 4832 time to create 1 rle with old method : 0.006011009216308594 length of segment : 103 time for calcul the mask position with numpy : 0.002365589141845703 nb_pixel_total : 68303 time to create 1 rle with old method : 0.07500243186950684 length of segment : 474 time for calcul the mask position with numpy : 0.0007607936859130859 nb_pixel_total : 15595 time to create 1 rle with old method : 0.01771998405456543 length of segment : 269 time for calcul the mask position with numpy : 0.0002143383026123047 nb_pixel_total : 5001 time to create 1 rle with old method : 0.005953311920166016 length of segment : 63 time for calcul the mask position with numpy : 0.00036835670471191406 nb_pixel_total : 6415 time to create 1 rle with old method : 0.007413625717163086 length of segment : 170 time for calcul the mask position with numpy : 0.00025463104248046875 nb_pixel_total : 2693 time to create 1 rle with old method : 0.003615856170654297 length of segment : 144 time for calcul the mask position with numpy : 0.003529787063598633 nb_pixel_total : 136605 time to create 1 rle with old method : 0.15293264389038086 length of segment : 776 time for calcul the mask position with numpy : 0.0015528202056884766 nb_pixel_total : 76403 time to create 1 rle with old method : 0.08513832092285156 length of segment : 244 time for calcul the mask position with numpy : 0.004107236862182617 nb_pixel_total : 185617 time to create 1 rle with new method : 0.00870656967163086 length of segment : 535 time for calcul the mask position with numpy : 0.00035691261291503906 nb_pixel_total : 4933 time to create 1 rle with old method : 0.0061283111572265625 length of segment : 115 time for calcul the mask position with numpy : 0.0009157657623291016 nb_pixel_total : 73564 time to create 1 rle with old method : 0.08272314071655273 length of segment : 206 time for calcul the mask position with numpy : 0.001589059829711914 nb_pixel_total : 42447 time to create 1 rle with old method : 0.04726123809814453 length of segment : 464 time for calcul the mask position with numpy : 0.007012605667114258 nb_pixel_total : 276356 time to create 1 rle with new method : 0.019753694534301758 length of segment : 762 time for calcul the mask position with numpy : 0.0008208751678466797 nb_pixel_total : 21060 time to create 1 rle with old method : 0.027242660522460938 length of segment : 233 time for calcul the mask position with numpy : 0.0003898143768310547 nb_pixel_total : 16126 time to create 1 rle with old method : 0.018172502517700195 length of segment : 175 time for calcul the mask position with numpy : 0.0003209114074707031 nb_pixel_total : 6433 time to create 1 rle with old method : 0.007673740386962891 length of segment : 120 time for calcul the mask position with numpy : 0.0008912086486816406 nb_pixel_total : 22922 time to create 1 rle with old method : 0.027018308639526367 length of segment : 230 time for calcul the mask position with numpy : 0.0015742778778076172 nb_pixel_total : 38319 time to create 1 rle with old method : 0.06147909164428711 length of segment : 339 time for calcul the mask position with numpy : 0.0003437995910644531 nb_pixel_total : 6700 time to create 1 rle with old method : 0.00817561149597168 length of segment : 126 time for calcul the mask position with numpy : 0.0003364086151123047 nb_pixel_total : 7419 time to create 1 rle with old method : 0.00968480110168457 length of segment : 88 time for calcul the mask position with numpy : 0.0004220008850097656 nb_pixel_total : 12512 time to create 1 rle with old method : 0.014580249786376953 length of segment : 147 time for calcul the mask position with numpy : 0.0009665489196777344 nb_pixel_total : 39726 time to create 1 rle with old method : 0.04616713523864746 length of segment : 325 time for calcul the mask position with numpy : 0.0009891986846923828 nb_pixel_total : 36319 time to create 1 rle with old method : 0.0404515266418457 length of segment : 260 time for calcul the mask position with numpy : 0.00029778480529785156 nb_pixel_total : 7081 time to create 1 rle with old method : 0.008239030838012695 length of segment : 123 time for calcul the mask position with numpy : 0.002402782440185547 nb_pixel_total : 53945 time to create 1 rle with old method : 0.06257462501525879 length of segment : 419 time for calcul the mask position with numpy : 0.0010933876037597656 nb_pixel_total : 39681 time to create 1 rle with old method : 0.044405221939086914 length of segment : 257 time for calcul the mask position with numpy : 0.001280069351196289 nb_pixel_total : 46355 time to create 1 rle with old method : 0.05435323715209961 length of segment : 223 time for calcul the mask position with numpy : 0.0012502670288085938 nb_pixel_total : 56745 time to create 1 rle with old method : 0.0664527416229248 length of segment : 295 time for calcul the mask position with numpy : 0.00058746337890625 nb_pixel_total : 5828 time to create 1 rle with old method : 0.0070683956146240234 length of segment : 238 time for calcul the mask position with numpy : 0.0010600090026855469 nb_pixel_total : 66060 time to create 1 rle with old method : 0.07754302024841309 length of segment : 278 time for calcul the mask position with numpy : 0.0007572174072265625 nb_pixel_total : 27174 time to create 1 rle with old method : 0.03321576118469238 length of segment : 234 time for calcul the mask position with numpy : 0.23468446731567383 nb_pixel_total : 1168457 time to create 1 rle with new method : 0.3813893795013428 length of segment : 2541 time for calcul the mask position with numpy : 0.0002446174621582031 nb_pixel_total : 3994 time to create 1 rle with old method : 0.004720926284790039 length of segment : 94 time for calcul the mask position with numpy : 0.014785528182983398 nb_pixel_total : 563769 time to create 1 rle with new method : 0.03488445281982422 length of segment : 1368 time for calcul the mask position with numpy : 0.0004029273986816406 nb_pixel_total : 11062 time to create 1 rle with old method : 0.013158321380615234 length of segment : 163 time for calcul the mask position with numpy : 0.0002181529998779297 nb_pixel_total : 7634 time to create 1 rle with old method : 0.009279012680053711 length of segment : 105 time for calcul the mask position with numpy : 0.0003173351287841797 nb_pixel_total : 19589 time to create 1 rle with old method : 0.023000478744506836 length of segment : 151 time for calcul the mask position with numpy : 0.00023031234741210938 nb_pixel_total : 3269 time to create 1 rle with old method : 0.003885984420776367 length of segment : 114 time for calcul the mask position with numpy : 0.00042819976806640625 nb_pixel_total : 8910 time to create 1 rle with old method : 0.011050224304199219 length of segment : 149 time for calcul the mask position with numpy : 0.0012054443359375 nb_pixel_total : 61935 time to create 1 rle with old method : 0.07007718086242676 length of segment : 425 time for calcul the mask position with numpy : 0.0035653114318847656 nb_pixel_total : 118254 time to create 1 rle with old method : 0.146651029586792 length of segment : 747 time for calcul the mask position with numpy : 0.00024318695068359375 nb_pixel_total : 1717 time to create 1 rle with old method : 0.0022068023681640625 length of segment : 70 time for calcul the mask position with numpy : 0.0009632110595703125 nb_pixel_total : 34391 time to create 1 rle with old method : 0.040679216384887695 length of segment : 280 time for calcul the mask position with numpy : 0.0020220279693603516 nb_pixel_total : 48738 time to create 1 rle with old method : 0.05721473693847656 length of segment : 344 time for calcul the mask position with numpy : 0.0007791519165039062 nb_pixel_total : 25513 time to create 1 rle with old method : 0.030225276947021484 length of segment : 189 time for calcul the mask position with numpy : 0.0005357265472412109 nb_pixel_total : 22961 time to create 1 rle with old method : 0.027014970779418945 length of segment : 177 time for calcul the mask position with numpy : 0.0003857612609863281 nb_pixel_total : 11680 time to create 1 rle with old method : 0.014182567596435547 length of segment : 108 time for calcul the mask position with numpy : 0.00040435791015625 nb_pixel_total : 15311 time to create 1 rle with old method : 0.018474340438842773 length of segment : 151 time for calcul the mask position with numpy : 0.0016963481903076172 nb_pixel_total : 88694 time to create 1 rle with old method : 0.10616731643676758 length of segment : 424 time for calcul the mask position with numpy : 0.0017771720886230469 nb_pixel_total : 28448 time to create 1 rle with old method : 0.04408621788024902 length of segment : 206 time for calcul the mask position with numpy : 0.061092376708984375 nb_pixel_total : 1477745 time to create 1 rle with new method : 0.39181017875671387 length of segment : 2084 time for calcul the mask position with numpy : 9.632110595703125e-05 nb_pixel_total : 1866 time to create 1 rle with old method : 0.0024225711822509766 length of segment : 32 time for calcul the mask position with numpy : 0.0004000663757324219 nb_pixel_total : 12201 time to create 1 rle with old method : 0.014278411865234375 length of segment : 165 time for calcul the mask position with numpy : 0.007825851440429688 nb_pixel_total : 392939 time to create 1 rle with new method : 0.07855606079101562 length of segment : 726 time for calcul the mask position with numpy : 0.0005249977111816406 nb_pixel_total : 9384 time to create 1 rle with old method : 0.011595010757446289 length of segment : 185 time for calcul the mask position with numpy : 0.00036787986755371094 nb_pixel_total : 12751 time to create 1 rle with old method : 0.016383886337280273 length of segment : 123 time for calcul the mask position with numpy : 0.00063323974609375 nb_pixel_total : 36183 time to create 1 rle with old method : 0.04488968849182129 length of segment : 250 time for calcul the mask position with numpy : 0.0009098052978515625 nb_pixel_total : 27489 time to create 1 rle with old method : 0.03591156005859375 length of segment : 146 time for calcul the mask position with numpy : 0.00016880035400390625 nb_pixel_total : 8179 time to create 1 rle with old method : 0.010116100311279297 length of segment : 126 time for calcul the mask position with numpy : 7.939338684082031e-05 nb_pixel_total : 1590 time to create 1 rle with old method : 0.002042055130004883 length of segment : 60 time for calcul the mask position with numpy : 0.0016057491302490234 nb_pixel_total : 64394 time to create 1 rle with old method : 0.07950973510742188 length of segment : 341 time spent for convertir_results : 15.706891775131226 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 151 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 48040 save missing photos in datou_result : time spend for datou_step_exec : 71.49461579322815 time spend to save output : 5.771780729293823 total time spend for step 1 : 77.26639652252197 step2:crop_condition Mon Feb 10 03:05:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 151 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 12 About to insert : list_path_to_insert length 12 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 46 About to insert : list_path_to_insert length 46 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335460491, 1335460487, 1335460482, 1335460478, 1335460424, 1335460418, 1335460414, 1335460401, 1335460398] Looping around the photos to save general results len do output : 85 /-3660544932Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544950Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544943Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544969Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559492Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559525Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544938Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544940Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544933Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544936Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544937Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559477Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559518Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559515Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544955Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544974Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660545001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660545012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559499Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544942Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544939Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3667061668Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544948Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3667061669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544949Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544951Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544952Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544946Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544963Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544973Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544975Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544977Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544965Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544968Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544979Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544971Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544959Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544980Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660545009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660545015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660545007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660545004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559482Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559480Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559476Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559506Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559517Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559512Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559529Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559526Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544934Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544956Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544947Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544964Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660544972Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559497Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559521Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559491Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559500Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3660559527Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460491', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460487', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460482', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460478', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460424', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460418', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460414', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460401', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460398', None, None, None, None, None, '2569872') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 264 time used for this insertion : 0.03817558288574219 save_final save missing photos in datou_result : time spend for datou_step_exec : 42.58229327201843 time spend to save output : 0.0810701847076416 total time spend for step 2 : 42.663363456726074 step3:thcl Mon Feb 10 03:06:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.005790233612060547 time to convert the images to numpy array : 0.01956629753112793 time to import caffe and check if the image exist : 0.007049560546875 time to convert the images to numpy array : 0.3518552780151367 time to import caffe and check if the image exist : 0.015375375747680664 time to convert the images to numpy array : 0.3482811450958252 time to import caffe and check if the image exist : 0.017354726791381836 time to convert the images to numpy array : 0.34804821014404297 time to import caffe and check if the image exist : 0.014591693878173828 time to convert the images to numpy array : 0.3567829132080078 time to import caffe and check if the image exist : 0.01377105712890625 time to convert the images to numpy array : 0.3596975803375244 time to import caffe and check if the image exist : 0.011338949203491211 time to convert the images to numpy array : 0.3813316822052002 time to import caffe and check if the image exist : 0.013411760330200195 time to convert the images to numpy array : 0.3843505382537842 time to import caffe and check if the image exist : 0.012343406677246094 time to convert the images to numpy array : 0.3909473419189453 time to import caffe and check if the image exist : 0.012789249420166016 time to convert the images to numpy array : 0.4088442325592041 total time to convert the images to numpy array : 0.6256742477416992 list photo_ids error: [] list photo_ids correct : [-3660559521, -3660559491, -3660559500, -3660559527, -3660544932, -3660544950, -3660544943, -3660544969, -3660559474, -3660559495, -3660559492, -3660559525, -3660544931, -3660559526, -3660544934, -3660544956, -3660544947, -3660544964, -3660544972, -3660559494, -3660559497, -3660559505, -3660559509, -3660559515, -3660544955, -3660544974, -3660544987, -3660544982, -3660545001, -3660545012, -3660559499, -3660559478, -3660559487, -3660559481, -3660559496, -3660559510, -3660559506, -3660559517, -3660559512, -3660559529, -3660544983, -3660544988, -3660544963, -3660544973, -3660544975, -3660544977, -3660544965, -3660544968, -3660544985, -3660545009, -3660545015, -3660545007, -3660545004, -3660559482, -3660559480, -3660559471, -3660559484, -3660559476, -3660544986, -3660544979, -3660544971, -3660544959, -3660544980, -3660544991, -3660544990, -3660544989, -3660544992, -3660544942, -3660544939, -3667061668, -3660544948, -3667061669, -3660544949, -3660544951, -3660544952, -3660544946, -3660544938, -3660544940, -3660544933, -3660544936, -3660544937, -3660544998, -3660544997, -3660559477, -3660559518] number of photos to traite : 85 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4822 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4603 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0862257480621338 time used to do the prediction : 0.19765138626098633 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4601 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4601 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.2546393871307373 time used to do the prediction : 0.1766200065612793 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.6450881958007812e-05 save missing photos in datou_result : time spend for datou_step_exec : 10.362947940826416 time spend to save output : 0.0009827613830566406 total time spend for step 3 : 10.363930702209473 step4:merge_mask_thcl_custom Mon Feb 10 03:06:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 151 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1335460491, 1335460487, 1335460482, 1335460478, 1335460424, 1335460418, 1335460414, 1335460401, 1335460398] Looping around the photos to save general results len do output : 9 /1335460491Didn't retrieve data .Didn't retrieve data . /1335460487Didn't retrieve data .Didn't retrieve data . /1335460482Didn't retrieve data .Didn't retrieve data . /1335460478Didn't retrieve data .Didn't retrieve data . /1335460424Didn't retrieve data .Didn't retrieve data . /1335460418Didn't retrieve data .Didn't retrieve data . /1335460414Didn't retrieve data .Didn't retrieve data . /1335460401Didn't retrieve data .Didn't retrieve data . /1335460398Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460491', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460487', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460482', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460478', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460424', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460418', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460414', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460401', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460398', None, None, None, None, None, '2569872') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.014989376068115234 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06687092781066895 time spend to save output : 0.015531301498413086 total time spend for step 4 : 0.08240222930908203 step5:rle_unique_nms_with_priority Mon Feb 10 03:06:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 9.078514337539673 time for calcul the mask position with numpy : 0.2542259693145752 nb_pixel_total : 5710765 time to create 1 rle with new method : 0.38892698287963867 time for calcul the mask position with numpy : 0.03439927101135254 nb_pixel_total : 15985 time to create 1 rle with old method : 0.01795482635498047 time for calcul the mask position with numpy : 0.026911497116088867 nb_pixel_total : 56848 time to create 1 rle with old method : 0.06340670585632324 time for calcul the mask position with numpy : 0.02703237533569336 nb_pixel_total : 35962 time to create 1 rle with old method : 0.040209293365478516 time for calcul the mask position with numpy : 0.025655508041381836 nb_pixel_total : 25119 time to create 1 rle with old method : 0.02766275405883789 time for calcul the mask position with numpy : 0.026905298233032227 nb_pixel_total : 6997 time to create 1 rle with old method : 0.007987022399902344 time for calcul the mask position with numpy : 0.028986454010009766 nb_pixel_total : 25778 time to create 1 rle with old method : 0.02903461456298828 time for calcul the mask position with numpy : 0.03093409538269043 nb_pixel_total : 156659 time to create 1 rle with new method : 0.38431477546691895 time for calcul the mask position with numpy : 0.02927994728088379 nb_pixel_total : 179157 time to create 1 rle with new method : 0.42824745178222656 time for calcul the mask position with numpy : 0.031224489212036133 nb_pixel_total : 583568 time to create 1 rle with new method : 0.5030951499938965 time for calcul the mask position with numpy : 0.02432870864868164 nb_pixel_total : 115470 time to create 1 rle with old method : 0.13493037223815918 time for calcul the mask position with numpy : 0.02276468276977539 nb_pixel_total : 91093 time to create 1 rle with old method : 0.11193251609802246 time for calcul the mask position with numpy : 0.037259817123413086 nb_pixel_total : 46839 time to create 1 rle with old method : 0.05118107795715332 create new chi : 2.901643753051758 time to delete rle : 0.014545917510986328 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 13827 TO DO : save crop sub photo not yet done ! save time : 1.6998281478881836 nb_obj : 16 nb_hashtags : 5 time to prepare the origin masks : 9.289061546325684 time for calcul the mask position with numpy : 0.3157823085784912 nb_pixel_total : 6259674 time to create 1 rle with new method : 0.6661131381988525 time for calcul the mask position with numpy : 0.050446510314941406 nb_pixel_total : 15103 time to create 1 rle with old method : 0.016496658325195312 time for calcul the mask position with numpy : 0.05060553550720215 nb_pixel_total : 47133 time to create 1 rle with old method : 0.05566763877868652 time for calcul the mask position with numpy : 0.05100584030151367 nb_pixel_total : 7 time to create 1 rle with old method : 4.744529724121094e-05 time for calcul the mask position with numpy : 0.028842926025390625 nb_pixel_total : 53896 time to create 1 rle with old method : 0.07022786140441895 time for calcul the mask position with numpy : 0.03255748748779297 nb_pixel_total : 63580 time to create 1 rle with old method : 0.0794668197631836 time for calcul the mask position with numpy : 0.04314875602722168 nb_pixel_total : 58384 time to create 1 rle with old method : 0.06078815460205078 time for calcul the mask position with numpy : 0.04146623611450195 nb_pixel_total : 170954 time to create 1 rle with new method : 0.5518946647644043 time for calcul the mask position with numpy : 0.04104185104370117 nb_pixel_total : 2073 time to create 1 rle with old method : 0.0026488304138183594 time for calcul the mask position with numpy : 0.044773101806640625 nb_pixel_total : 46442 time to create 1 rle with old method : 0.05237531661987305 time for calcul the mask position with numpy : 0.038631439208984375 nb_pixel_total : 36463 time to create 1 rle with old method : 0.03936409950256348 time for calcul the mask position with numpy : 0.04351449012756348 nb_pixel_total : 7201 time to create 1 rle with old method : 0.009540081024169922 time for calcul the mask position with numpy : 0.049111366271972656 nb_pixel_total : 83941 time to create 1 rle with old method : 0.09313082695007324 time for calcul the mask position with numpy : 0.04164695739746094 nb_pixel_total : 13695 time to create 1 rle with old method : 0.014918804168701172 time for calcul the mask position with numpy : 0.036745309829711914 nb_pixel_total : 3107 time to create 1 rle with old method : 0.003522157669067383 time for calcul the mask position with numpy : 0.024406909942626953 nb_pixel_total : 41487 time to create 1 rle with old method : 0.05300188064575195 time for calcul the mask position with numpy : 0.027196168899536133 nb_pixel_total : 147100 time to create 1 rle with old method : 0.15524578094482422 create new chi : 2.9664227962493896 time to delete rle : 0.0011632442474365234 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 10537 TO DO : save crop sub photo not yet done ! save time : 1.3983092308044434 nb_obj : 29 nb_hashtags : 6 time to prepare the origin masks : 14.454516649246216 time for calcul the mask position with numpy : 1.0436832904815674 nb_pixel_total : 5346471 time to create 1 rle with new method : 0.7912144660949707 time for calcul the mask position with numpy : 0.03341484069824219 nb_pixel_total : 5400 time to create 1 rle with old method : 0.008057355880737305 time for calcul the mask position with numpy : 0.03569769859313965 nb_pixel_total : 191964 time to create 1 rle with new method : 0.5690338611602783 time for calcul the mask position with numpy : 0.028720855712890625 nb_pixel_total : 2123 time to create 1 rle with old method : 0.0025353431701660156 time for calcul the mask position with numpy : 0.03005504608154297 nb_pixel_total : 63774 time to create 1 rle with old method : 0.08114242553710938 time for calcul the mask position with numpy : 0.04795479774475098 nb_pixel_total : 3663 time to create 1 rle with old method : 0.006068229675292969 time for calcul the mask position with numpy : 0.033776283264160156 nb_pixel_total : 70143 time to create 1 rle with old method : 0.07734870910644531 time for calcul the mask position with numpy : 0.029998779296875 nb_pixel_total : 25071 time to create 1 rle with old method : 0.03518366813659668 time for calcul the mask position with numpy : 0.033594369888305664 nb_pixel_total : 23327 time to create 1 rle with old method : 0.027376890182495117 time for calcul the mask position with numpy : 0.03228592872619629 nb_pixel_total : 150161 time to create 1 rle with new method : 0.3755943775177002 time for calcul the mask position with numpy : 0.035645246505737305 nb_pixel_total : 6139 time to create 1 rle with old method : 0.010115385055541992 time for calcul the mask position with numpy : 0.03324317932128906 nb_pixel_total : 19010 time to create 1 rle with old method : 0.02283310890197754 time for calcul the mask position with numpy : 0.031577348709106445 nb_pixel_total : 6419 time to create 1 rle with old method : 0.034474849700927734 time for calcul the mask position with numpy : 0.030426740646362305 nb_pixel_total : 72284 time to create 1 rle with old method : 0.08499002456665039 time for calcul the mask position with numpy : 0.03282642364501953 nb_pixel_total : 65733 time to create 1 rle with old method : 0.07330560684204102 time for calcul the mask position with numpy : 0.032556772232055664 nb_pixel_total : 86775 time to create 1 rle with old method : 0.09505748748779297 time for calcul the mask position with numpy : 0.029959440231323242 nb_pixel_total : 41384 time to create 1 rle with old method : 0.04537057876586914 time for calcul the mask position with numpy : 0.029675006866455078 nb_pixel_total : 1097 time to create 1 rle with old method : 0.0014138221740722656 time for calcul the mask position with numpy : 0.02748870849609375 nb_pixel_total : 1587 time to create 1 rle with old method : 0.0018126964569091797 time for calcul the mask position with numpy : 0.027936458587646484 nb_pixel_total : 12889 time to create 1 rle with old method : 0.013659477233886719 time for calcul the mask position with numpy : 0.029490232467651367 nb_pixel_total : 190049 time to create 1 rle with new method : 0.8929860591888428 time for calcul the mask position with numpy : 0.02872443199157715 nb_pixel_total : 82392 time to create 1 rle with old method : 0.0869135856628418 time for calcul the mask position with numpy : 0.029500961303710938 nb_pixel_total : 139083 time to create 1 rle with old method : 0.14160728454589844 time for calcul the mask position with numpy : 0.02916574478149414 nb_pixel_total : 27921 time to create 1 rle with old method : 0.030750751495361328 time for calcul the mask position with numpy : 0.029453754425048828 nb_pixel_total : 3368 time to create 1 rle with old method : 0.004938840866088867 time for calcul the mask position with numpy : 0.03314518928527832 nb_pixel_total : 33647 time to create 1 rle with old method : 0.03451180458068848 time for calcul the mask position with numpy : 0.03215217590332031 nb_pixel_total : 323419 time to create 1 rle with new method : 0.518444299697876 time for calcul the mask position with numpy : 0.02787041664123535 nb_pixel_total : 3608 time to create 1 rle with old method : 0.004042863845825195 time for calcul the mask position with numpy : 0.02831721305847168 nb_pixel_total : 43400 time to create 1 rle with old method : 0.045534372329711914 time for calcul the mask position with numpy : 0.028826236724853516 nb_pixel_total : 7939 time to create 1 rle with old method : 0.008894205093383789 create new chi : 6.2363526821136475 time to delete rle : 0.0024251937866210938 batch 1 Loaded 30 chid ids of type : 3726 Number RLEs to save : 17867 TO DO : save crop sub photo not yet done ! save time : 1.9171898365020752 nb_obj : 13 nb_hashtags : 5 time to prepare the origin masks : 11.926186323165894 time for calcul the mask position with numpy : 1.728646993637085 nb_pixel_total : 6224876 time to create 1 rle with new method : 0.9954955577850342 time for calcul the mask position with numpy : 0.03824973106384277 nb_pixel_total : 44199 time to create 1 rle with old method : 0.047005653381347656 time for calcul the mask position with numpy : 0.04300665855407715 nb_pixel_total : 84213 time to create 1 rle with old method : 0.0888204574584961 time for calcul the mask position with numpy : 0.03736615180969238 nb_pixel_total : 2796 time to create 1 rle with old method : 0.002954721450805664 time for calcul the mask position with numpy : 0.037507057189941406 nb_pixel_total : 28583 time to create 1 rle with old method : 0.029795169830322266 time for calcul the mask position with numpy : 0.03863215446472168 nb_pixel_total : 2528 time to create 1 rle with old method : 0.0029191970825195312 time for calcul the mask position with numpy : 0.03927421569824219 nb_pixel_total : 6343 time to create 1 rle with old method : 0.0070209503173828125 time for calcul the mask position with numpy : 0.03878021240234375 nb_pixel_total : 23085 time to create 1 rle with old method : 0.027151823043823242 time for calcul the mask position with numpy : 0.04029989242553711 nb_pixel_total : 112089 time to create 1 rle with old method : 0.12353396415710449 time for calcul the mask position with numpy : 0.03769516944885254 nb_pixel_total : 2048 time to create 1 rle with old method : 0.002210855484008789 time for calcul the mask position with numpy : 0.04201817512512207 nb_pixel_total : 8129 time to create 1 rle with old method : 0.008838415145874023 time for calcul the mask position with numpy : 0.03452253341674805 nb_pixel_total : 86793 time to create 1 rle with old method : 0.08956503868103027 time for calcul the mask position with numpy : 0.03382682800292969 nb_pixel_total : 78269 time to create 1 rle with old method : 0.10024428367614746 time for calcul the mask position with numpy : 0.05279421806335449 nb_pixel_total : 346289 time to create 1 rle with new method : 0.5412142276763916 create new chi : 4.378256559371948 time to delete rle : 0.0010218620300292969 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 9901 TO DO : save crop sub photo not yet done ! save time : 0.710230827331543 nb_obj : 14 nb_hashtags : 4 time to prepare the origin masks : 12.071892261505127 time for calcul the mask position with numpy : 0.7993979454040527 nb_pixel_total : 6701529 time to create 1 rle with new method : 1.2081429958343506 time for calcul the mask position with numpy : 0.047615766525268555 nb_pixel_total : 33074 time to create 1 rle with old method : 0.040743112564086914 time for calcul the mask position with numpy : 0.04834318161010742 nb_pixel_total : 18592 time to create 1 rle with old method : 0.025854110717773438 time for calcul the mask position with numpy : 0.04007720947265625 nb_pixel_total : 5754 time to create 1 rle with old method : 0.00632786750793457 time for calcul the mask position with numpy : 0.02826094627380371 nb_pixel_total : 5564 time to create 1 rle with old method : 0.006356000900268555 time for calcul the mask position with numpy : 0.027602434158325195 nb_pixel_total : 41255 time to create 1 rle with old method : 0.04250741004943848 time for calcul the mask position with numpy : 0.04119229316711426 nb_pixel_total : 11011 time to create 1 rle with old method : 0.013006925582885742 time for calcul the mask position with numpy : 0.04151463508605957 nb_pixel_total : 2489 time to create 1 rle with old method : 0.004011869430541992 time for calcul the mask position with numpy : 0.0684061050415039 nb_pixel_total : 11934 time to create 1 rle with old method : 0.015729904174804688 time for calcul the mask position with numpy : 0.06417393684387207 nb_pixel_total : 24893 time to create 1 rle with old method : 0.05141448974609375 time for calcul the mask position with numpy : 0.04599142074584961 nb_pixel_total : 7257 time to create 1 rle with old method : 0.009013652801513672 time for calcul the mask position with numpy : 0.03509998321533203 nb_pixel_total : 138783 time to create 1 rle with old method : 0.15996766090393066 time for calcul the mask position with numpy : 0.02512049674987793 nb_pixel_total : 3315 time to create 1 rle with old method : 0.004573822021484375 time for calcul the mask position with numpy : 0.022862672805786133 nb_pixel_total : 17018 time to create 1 rle with old method : 0.021989107131958008 time for calcul the mask position with numpy : 0.023596525192260742 nb_pixel_total : 27772 time to create 1 rle with old method : 0.039165496826171875 create new chi : 3.061373710632324 time to delete rle : 0.0010824203491210938 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 7136 TO DO : save crop sub photo not yet done ! save time : 1.9415252208709717 nb_obj : 16 nb_hashtags : 5 time to prepare the origin masks : 11.668885707855225 time for calcul the mask position with numpy : 0.23943829536437988 nb_pixel_total : 6129138 time to create 1 rle with new method : 1.3918225765228271 time for calcul the mask position with numpy : 0.027150630950927734 nb_pixel_total : 136328 time to create 1 rle with old method : 0.1634964942932129 time for calcul the mask position with numpy : 0.046279191970825195 nb_pixel_total : 15373 time to create 1 rle with old method : 0.02289128303527832 time for calcul the mask position with numpy : 0.04846668243408203 nb_pixel_total : 22801 time to create 1 rle with old method : 0.025304317474365234 time for calcul the mask position with numpy : 0.04193687438964844 nb_pixel_total : 38266 time to create 1 rle with old method : 0.041863203048706055 time for calcul the mask position with numpy : 0.04238748550415039 nb_pixel_total : 68206 time to create 1 rle with old method : 0.0704946517944336 time for calcul the mask position with numpy : 0.04229164123535156 nb_pixel_total : 20902 time to create 1 rle with old method : 0.022205114364624023 time for calcul the mask position with numpy : 0.03715872764587402 nb_pixel_total : 42176 time to create 1 rle with old method : 0.04715299606323242 time for calcul the mask position with numpy : 0.05167579650878906 nb_pixel_total : 7349 time to create 1 rle with old method : 0.009331941604614258 time for calcul the mask position with numpy : 0.047410011291503906 nb_pixel_total : 4924 time to create 1 rle with old method : 0.007523775100708008 time for calcul the mask position with numpy : 0.04994988441467285 nb_pixel_total : 73538 time to create 1 rle with old method : 0.07741832733154297 time for calcul the mask position with numpy : 0.04035592079162598 nb_pixel_total : 276179 time to create 1 rle with new method : 2.0758965015411377 time for calcul the mask position with numpy : 0.042038679122924805 nb_pixel_total : 2680 time to create 1 rle with old method : 0.003339529037475586 time for calcul the mask position with numpy : 0.04145407676696777 nb_pixel_total : 4970 time to create 1 rle with old method : 0.006191730499267578 time for calcul the mask position with numpy : 0.05002331733703613 nb_pixel_total : 15546 time to create 1 rle with old method : 0.019721031188964844 time for calcul the mask position with numpy : 0.02741718292236328 nb_pixel_total : 6409 time to create 1 rle with old method : 0.010171890258789062 time for calcul the mask position with numpy : 0.031017065048217773 nb_pixel_total : 185455 time to create 1 rle with new method : 0.6393280029296875 create new chi : 5.6437389850616455 time to delete rle : 0.0018115043640136719 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 12218 TO DO : save crop sub photo not yet done ! save time : 1.3534319400787354 nb_obj : 13 nb_hashtags : 6 time to prepare the origin masks : 12.707730054855347 time for calcul the mask position with numpy : 0.9569318294525146 nb_pixel_total : 6702240 time to create 1 rle with new method : 1.5435214042663574 time for calcul the mask position with numpy : 0.026269197463989258 nb_pixel_total : 329 time to create 1 rle with old method : 0.0006351470947265625 time for calcul the mask position with numpy : 0.04336118698120117 nb_pixel_total : 53655 time to create 1 rle with old method : 0.06941509246826172 time for calcul the mask position with numpy : 0.048426151275634766 nb_pixel_total : 27068 time to create 1 rle with old method : 0.03217053413391113 time for calcul the mask position with numpy : 0.04066753387451172 nb_pixel_total : 65920 time to create 1 rle with old method : 0.07061004638671875 time for calcul the mask position with numpy : 0.04108476638793945 nb_pixel_total : 5803 time to create 1 rle with old method : 0.007831573486328125 time for calcul the mask position with numpy : 0.0454099178314209 nb_pixel_total : 39656 time to create 1 rle with old method : 0.05229616165161133 time for calcul the mask position with numpy : 0.050272226333618164 nb_pixel_total : 46202 time to create 1 rle with old method : 0.05633234977722168 time for calcul the mask position with numpy : 0.027521371841430664 nb_pixel_total : 36264 time to create 1 rle with old method : 0.04941511154174805 time for calcul the mask position with numpy : 0.0404515266418457 nb_pixel_total : 7058 time to create 1 rle with old method : 0.009194612503051758 time for calcul the mask position with numpy : 0.05069470405578613 nb_pixel_total : 12474 time to create 1 rle with old method : 0.01351618766784668 time for calcul the mask position with numpy : 0.0370180606842041 nb_pixel_total : 39582 time to create 1 rle with old method : 0.040807247161865234 time for calcul the mask position with numpy : 0.038689374923706055 nb_pixel_total : 7307 time to create 1 rle with old method : 0.008330345153808594 time for calcul the mask position with numpy : 0.040879249572753906 nb_pixel_total : 6682 time to create 1 rle with old method : 0.009553194046020508 create new chi : 3.5083680152893066 time to delete rle : 0.0013890266418457031 batch 1 Loaded 14 chid ids of type : 3726 Number RLEs to save : 7735 TO DO : save crop sub photo not yet done ! save time : 0.5631983280181885 nb_obj : 15 nb_hashtags : 5 time to prepare the origin masks : 8.755813121795654 time for calcul the mask position with numpy : 0.24358153343200684 nb_pixel_total : 4922865 time to create 1 rle with new method : 0.579625129699707 time for calcul the mask position with numpy : 0.0254819393157959 nb_pixel_total : 117413 time to create 1 rle with old method : 0.1232907772064209 time for calcul the mask position with numpy : 0.029577016830444336 nb_pixel_total : 22851 time to create 1 rle with old method : 0.024626731872558594 time for calcul the mask position with numpy : 0.023695707321166992 nb_pixel_total : 55417 time to create 1 rle with old method : 0.057558298110961914 time for calcul the mask position with numpy : 0.02245354652404785 nb_pixel_total : 15259 time to create 1 rle with old method : 0.016648530960083008 time for calcul the mask position with numpy : 0.02809000015258789 nb_pixel_total : 48456 time to create 1 rle with old method : 0.050374507904052734 time for calcul the mask position with numpy : 0.024456262588500977 nb_pixel_total : 1711 time to create 1 rle with old method : 0.0019032955169677734 time for calcul the mask position with numpy : 0.023586273193359375 nb_pixel_total : 34288 time to create 1 rle with old method : 0.036896467208862305 time for calcul the mask position with numpy : 0.025180578231811523 nb_pixel_total : 7599 time to create 1 rle with old method : 0.009079456329345703 time for calcul the mask position with numpy : 0.02344226837158203 nb_pixel_total : 8791 time to create 1 rle with old method : 0.009606599807739258 time for calcul the mask position with numpy : 0.022870779037475586 nb_pixel_total : 3257 time to create 1 rle with old method : 0.0035796165466308594 time for calcul the mask position with numpy : 0.023090839385986328 nb_pixel_total : 88536 time to create 1 rle with old method : 0.09244108200073242 time for calcul the mask position with numpy : 0.030601978302001953 nb_pixel_total : 11014 time to create 1 rle with old method : 0.012050151824951172 time for calcul the mask position with numpy : 0.05213737487792969 nb_pixel_total : 544609 time to create 1 rle with new method : 0.4620823860168457 time for calcul the mask position with numpy : 0.0382843017578125 nb_pixel_total : 3974 time to create 1 rle with old method : 0.004378557205200195 time for calcul the mask position with numpy : 0.05238223075866699 nb_pixel_total : 1164200 time to create 1 rle with new method : 0.3758225440979004 create new chi : 2.641225576400757 time to delete rle : 0.0015022754669189453 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 16041 TO DO : save crop sub photo not yet done ! save time : 1.824948787689209 nb_obj : 12 nb_hashtags : 5 time to prepare the origin masks : 9.655597925186157 time for calcul the mask position with numpy : 0.5091021060943604 nb_pixel_total : 4871337 time to create 1 rle with new method : 0.5265514850616455 time for calcul the mask position with numpy : 0.021373987197875977 nb_pixel_total : 64273 time to create 1 rle with old method : 0.0829935073852539 time for calcul the mask position with numpy : 0.023492813110351562 nb_pixel_total : 12178 time to create 1 rle with old method : 0.016726255416870117 time for calcul the mask position with numpy : 0.021271228790283203 nb_pixel_total : 1585 time to create 1 rle with old method : 0.10645890235900879 time for calcul the mask position with numpy : 0.02385401725769043 nb_pixel_total : 4708 time to create 1 rle with old method : 0.00519561767578125 time for calcul the mask position with numpy : 0.02323436737060547 nb_pixel_total : 12630 time to create 1 rle with old method : 0.01618051528930664 time for calcul the mask position with numpy : 0.02140069007873535 nb_pixel_total : 9358 time to create 1 rle with old method : 0.010338783264160156 time for calcul the mask position with numpy : 0.023617267608642578 nb_pixel_total : 351402 time to create 1 rle with new method : 0.8827934265136719 time for calcul the mask position with numpy : 0.02305436134338379 nb_pixel_total : 36046 time to create 1 rle with old method : 0.04187512397766113 time for calcul the mask position with numpy : 0.021816730499267578 nb_pixel_total : 1856 time to create 1 rle with old method : 0.002190828323364258 time for calcul the mask position with numpy : 0.022628068923950195 nb_pixel_total : 180951 time to create 1 rle with new method : 0.9635441303253174 time for calcul the mask position with numpy : 0.11111259460449219 nb_pixel_total : 1476534 time to create 1 rle with new method : 0.5217669010162354 time for calcul the mask position with numpy : 0.037390947341918945 nb_pixel_total : 27382 time to create 1 rle with old method : 0.03152179718017578 create new chi : 4.211882591247559 time to delete rle : 0.0015840530395507812 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 13594 TO DO : save crop sub photo not yet done ! save time : 0.7858202457427979 map_output_result : {1335460491: (0.0, 'Should be the crop_list due to order', 0.0), 1335460487: (0.0, 'Should be the crop_list due to order', 0.0), 1335460482: (0.0, 'Should be the crop_list due to order', 0.0), 1335460478: (0.0, 'Should be the crop_list due to order', 0.0), 1335460424: (0.0, 'Should be the crop_list due to order', 0.0), 1335460418: (0.0, 'Should be the crop_list due to order', 0.0), 1335460414: (0.0, 'Should be the crop_list due to order', 0.0), 1335460401: (0.0, 'Should be the crop_list due to order', 0.0), 1335460398: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1335460491, 1335460487, 1335460482, 1335460478, 1335460424, 1335460418, 1335460414, 1335460401, 1335460398] Looping around the photos to save general results len do output : 9 /1335460491.Didn't retrieve data . /1335460487.Didn't retrieve data . /1335460482.Didn't retrieve data . /1335460478.Didn't retrieve data . /1335460424.Didn't retrieve data . /1335460418.Didn't retrieve data . /1335460414.Didn't retrieve data . /1335460401.Didn't retrieve data . /1335460398.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460491', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460487', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460482', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460478', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460424', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460418', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460414', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460401', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460398', None, None, None, None, None, '2569872') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.014189004898071289 save_final save missing photos in datou_result : time spend for datou_step_exec : 148.3682837486267 time spend to save output : 0.01802682876586914 total time spend for step 5 : 148.38631057739258 step6:crop_condition Mon Feb 10 03:09:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 9 ! batch 1 Loaded 155 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739153350_826376 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3250272274017334 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739153357_826376 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3383240699768066 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739153362_826376 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.217860698699951 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739153366_826376 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.2455849647521973 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! About to upload 11 photos upload in portfolio : 4869462 init cache_photo without model_param we have 11 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739153372_826376 we have uploaded 11 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.845989942550659 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1739153377_826376 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.5358119010925293 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335460491, 1335460487, 1335460482, 1335460478, 1335460424, 1335460418, 1335460414, 1335460401, 1335460398] Looping around the photos to save general results len do output : 32 /1336356179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356180Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356181Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1336356233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460491', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460487', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460482', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460478', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460424', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460418', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460414', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460401', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460398', None, None, None, None, None, '2569872') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 105 time used for this insertion : 0.02675032615661621 save_final save missing photos in datou_result : time spend for datou_step_exec : 33.42456912994385 time spend to save output : 0.02825188636779785 total time spend for step 6 : 33.452821016311646 step7:ventilate_hashtags_in_portfolio Mon Feb 10 03:09:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20384176 get user id for portfolio 20384176 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20384176 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','papier','kraft','plastique','Carton_gris','autre_refus','mal_croppe','environnement','flou','Carton_brun','Teint_Dans_La_Masse','cartonnette')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20384176 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('metal','papier','kraft','plastique','Carton_gris','autre_refus','mal_croppe','environnement','flou','Carton_brun','Teint_Dans_La_Masse','cartonnette')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20384261,20384262,20384263,20384264,20384265,20384266,20384267,20384268,20384269,20384270,20384271,20384272?tags=environnement,mal_croppe,Carton_gris,metal,Carton_brun,kraft,papier,plastique,flou,cartonnette,Teint_Dans_La_Masse,autre_refus Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1335460491, 1335460487, 1335460482, 1335460478, 1335460424, 1335460418, 1335460414, 1335460401, 1335460398] Looping around the photos to save general results len do output : 1 /20384176. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460491', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460487', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460482', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460478', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460424', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460418', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460414', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460401', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460398', None, None, None, None, None, '2569872') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.13728833198547363 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.5096092224121094 time spend to save output : 0.13761448860168457 total time spend for step 7 : 2.647223711013794 step8:final Mon Feb 10 03:09:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1335460491: ('0.10306427815014783',), 1335460487: ('0.10306427815014783',), 1335460482: ('0.10306427815014783',), 1335460478: ('0.10306427815014783',), 1335460424: ('0.10306427815014783',), 1335460418: ('0.10306427815014783',), 1335460414: ('0.10306427815014783',), 1335460401: ('0.10306427815014783',), 1335460398: ('0.10306427815014783',)} new output for save of step final : {1335460491: ('0.10306427815014783',), 1335460487: ('0.10306427815014783',), 1335460482: ('0.10306427815014783',), 1335460478: ('0.10306427815014783',), 1335460424: ('0.10306427815014783',), 1335460418: ('0.10306427815014783',), 1335460414: ('0.10306427815014783',), 1335460401: ('0.10306427815014783',), 1335460398: ('0.10306427815014783',)} [1335460491, 1335460487, 1335460482, 1335460478, 1335460424, 1335460418, 1335460414, 1335460401, 1335460398] Looping around the photos to save general results len do output : 9 /1335460491.Didn't retrieve data . /1335460487.Didn't retrieve data . /1335460482.Didn't retrieve data . /1335460478.Didn't retrieve data . /1335460424.Didn't retrieve data . /1335460418.Didn't retrieve data . /1335460414.Didn't retrieve data . /1335460401.Didn't retrieve data . /1335460398.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460491', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460487', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460482', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460478', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460424', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460418', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460414', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460401', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460398', None, None, None, None, None, '2569872') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 27 time used for this insertion : 0.08844614028930664 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11940550804138184 time spend to save output : 0.08913350105285645 total time spend for step 8 : 0.20853900909423828 step9:velours_tree Mon Feb 10 03:09:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.1553785800933838 time spend to save output : 5.173683166503906e-05 total time spend for step 9 : 0.15543031692504883 step10:send_mail_cod Mon Feb 10 03:09:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20384176_10-02-2025_03_09_44.pdf 20384262 imagette203842621739153384 20384264 imagette203842641739153384 20384266 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203842661739153384 20384268 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203842681739153385 20384269 imagette203842691739153385 20384270 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203842701739153385 20384272 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203842721739153386 20384263 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203842631739153389 20384265 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203842651739153390 20384271 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203842711739153392 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20384176 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20384261,20384262,20384263,20384264,20384265,20384266,20384267,20384268,20384269,20384270,20384271,20384272?tags=environnement,mal_croppe,Carton_gris,metal,Carton_brun,kraft,papier,plastique,flou,cartonnette,Teint_Dans_La_Masse,autre_refus your option no_mail is active, we will not send the real mail to your client args[1335460491] : ('0.10306427815014783',) no score found for photo 1335460491 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com args[1335460487] : ('0.10306427815014783',) no score found for photo 1335460487 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com args[1335460482] : ('0.10306427815014783',) no score found for photo 1335460482 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com args[1335460478] : ('0.10306427815014783',) no score found for photo 1335460478 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com args[1335460424] : ('0.10306427815014783',) no score found for photo 1335460424 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com args[1335460418] : ('0.10306427815014783',) no score found for photo 1335460418 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com args[1335460414] : ('0.10306427815014783',) no score found for photo 1335460414 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com args[1335460401] : ('0.10306427815014783',) no score found for photo 1335460401 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com args[1335460398] : ('0.10306427815014783',) no score found for photo 1335460398 apple ('0.10306427815014783',) We are sending mail with results at report@fotonower.com refus_total : 0.10306427815014783 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20384176 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1335460401,1335750458,1335750456,1335750415,1335474142,1335474138,1335472873,1335472785,1335460491,1335460487,1335460482,1335460478,1335460424,1335460414,1335460398,1335750476,1335750497,1335472989,1335460418) Found this number of photos: 19 begin to download photo : 1335460401 begin to download photo : 1335474142 begin to download photo : 1335460491 begin to download photo : 1335460424 begin to download photo : 1335750497 download finish for photo 1335460401 begin to download photo : 1335750458 download finish for photo 1335460491 begin to download photo : 1335460487 download finish for photo 1335474142 begin to download photo : 1335474138 download finish for photo 1335460424 begin to download photo : 1335460414 download finish for photo 1335750497 begin to download photo : 1335472989 download finish for photo 1335750458 begin to download photo : 1335750456 download finish for photo 1335460414 begin to download photo : 1335460398 download finish for photo 1335474138 begin to download photo : 1335472873 download finish for photo 1335460487 begin to download photo : 1335460482 download finish for photo 1335472989 begin to download photo : 1335460418 download finish for photo 1335750456 begin to download photo : 1335750415 download finish for photo 1335460398 begin to download photo : 1335750476 download finish for photo 1335472873 begin to download photo : 1335472785 download finish for photo 1335750415 download finish for photo 1335460482 begin to download photo : 1335460478 download finish for photo 1335750476 download finish for photo 1335460418 download finish for photo 1335472785 download finish for photo 1335460478 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20384176_10-02-2025_03_09_44.pdf results_Auto_P20384176_10-02-2025_03_09_44.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20384176_10-02-2025_03_09_44.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20384176','results_Auto_P20384176_10-02-2025_03_09_44.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20384176_10-02-2025_03_09_44.pdf','pdf','','1.18','0.10306427815014783') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1335460491, 1335460487, 1335460482, 1335460478, 1335460424, 1335460418, 1335460414, 1335460401, 1335460398] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460491', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460487', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460482', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460478', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460424', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460418', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460414', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460401', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460398', None, None, None, None, None, '2569872') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 9 time used for this insertion : 0.013628005981445312 save_final save missing photos in datou_result : time spend for datou_step_exec : 12.520060777664185 time spend to save output : 0.01388406753540039 total time spend for step 10 : 12.533944845199585 step11:split_time_score Mon Feb 10 03:09:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('09', 19),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 07022025 20384176 Nombre de photos uploadées : 19 / 23040 (0%) 07022025 20384176 Nombre de photos taguées (types de déchets): 0 / 19 (0%) 07022025 20384176 Nombre de photos taguées (volume) : 0 / 19 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ??????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0009911060333251953 elapsed_time : insert_dashboard_record_day_entry 0.8818044662475586 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.20455292078149484 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20356992_08-02-2025_08_22_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20356992 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20356992 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354355 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20356993 order by id desc limit 1 Qualite : 0.0937534478258616 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20354357_08-02-2025_18_37_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354357 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20354357 AND mptpi.`type`=4207 To do Qualite : 0.05165069322458233 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20356994_08-02-2025_14_41_11.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20356994 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20356994 AND mptpi.`type`=4207 To do Qualite : 0.04956703005461016 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20354361_08-02-2025_16_33_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354361 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20354361 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384172 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384173 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384174 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384175 order by id desc limit 1 Qualite : 0.10305182757245573 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20384176_10-02-2025_03_09_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384176 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20384176 AND mptpi.`type`=3726 To do Qualite : 0.2385312928452032 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20351777_07-02-2025_23_09_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20351777 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20351777 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384177 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384178 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384179 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384180 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384181 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384182 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384183 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384184 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384185 order by id desc limit 1 Qualite : 0.08605458446438688 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20356996_08-02-2025_15_15_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20356996 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20356996 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384186 order by id desc limit 1 Qualite : 0.04449428467556786 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20356998_08-02-2025_10_28_27.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20356998 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20356998 AND mptpi.`type`=4207 To do Qualite : 0.07605261699755003 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20356999_08-02-2025_10_15_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20356999 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20356999 AND mptpi.`type`=4207 To do Qualite : 0.06910444792161492 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20354364_08-02-2025_15_37_36.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354364 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20354364 AND mptpi.`type`=4207 To do Qualite : 0.05273507048020844 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20354365_08-02-2025_16_29_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354365 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20354365 AND mptpi.`type`=4207 To do Qualite : 0.05565544109539141 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20384187_10-02-2025_03_02_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384187 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20384187 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384188 order by id desc limit 1 Qualite : 0.08501720192728651 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20384189_10-02-2025_03_09_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384189 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20384189 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20384190 order by id desc limit 1 Qualite : 0.23094291825526506 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20357000_08-02-2025_06_44_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20357000 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20357000 AND mptpi.`type`=3594 To do Qualite : 0.25143322150104874 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20357001_08-02-2025_06_21_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20357001 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20357001 AND mptpi.`type`=3594 To do Qualite : 0.17662439860203344 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20357002_08-02-2025_05_55_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20357002 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20357002 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20357003 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354369 order by id desc limit 1 Qualite : 0.22202820203441462 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20354370_08-02-2025_01_55_34.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354370 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20354370 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354371 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354372 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20357004 order by id desc limit 1 Qualite : 0.09528119695403824 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20357005_08-02-2025_09_14_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20357005 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20357005 AND mptpi.`type`=4207 To do Qualite : 0.03888751778420364 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20354375_08-02-2025_15_16_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20354375 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20354375 AND mptpi.`type`=4207 To do Qualite : 0.4211175932107529 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20351861_08-02-2025_05_04_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20351861 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20351861 AND mptpi.`type`=4200 To do Qualite : 0.08701264816490367 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20351862_07-02-2025_22_27_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20351862 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20351862 AND mptpi.`type`=3726 To do Qualite : 0.18687183807817992 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20351863_07-02-2025_22_32_02.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20351863 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20351863 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20351864 order by id desc limit 1 Qualite : 0.03599672022473725 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20351865_07-02-2025_23_21_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20351865 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20351865 AND mptpi.`type`=4207 To do Qualite : 0.11103449586161805 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20351866_08-02-2025_01_40_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20351866 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20351866 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'07022025': {'nb_upload': 19, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1335460491, 1335460487, 1335460482, 1335460478, 1335460424, 1335460418, 1335460414, 1335460401, 1335460398] Looping around the photos to save general results len do output : 1 /20384176Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460491', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460487', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460482', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460478', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460424', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460418', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460414', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460401', None, None, None, None, None, '2569872') ('3459', None, None, None, None, None, None, None, '2569872') ('3459', None, '1335460398', None, None, None, None, None, '2569872') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.013404369354248047 save_final save missing photos in datou_result : time spend for datou_step_exec : 21.343884706497192 time spend to save output : 0.013655900955200195 total time spend for step 11 : 21.357540607452393 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 9 set_done_treatment 148.24user 145.83system 5:54.71elapsed 82%CPU (0avgtext+0avgdata 4393364maxresident)k 2524976inputs+151520outputs (25873major+13140962minor)pagefaults 0swaps