python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4139576 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 26>10 please execute split_portfolio.py -i 20881947 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2619767'] with mtr_portfolio_ids : ['20881947'] and first list_photo_ids : [] new path : /proc/4139576/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.8692853450775146 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Feb 27 15:46:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-27 15:46:34.336492: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-27 15:46:34.371390: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-27 15:46:34.373100: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe920000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-27 15:46:34.373123: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-27 15:46:34.378539: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-27 15:46:34.722331: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xa225820 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-27 15:46:34.722391: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-27 15:46:34.723964: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 15:46:34.724733: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 15:46:34.757043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 15:46:34.771998: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 15:46:34.773549: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 15:46:34.781025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 15:46:34.783666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 15:46:34.803332: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 15:46:34.805174: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 15:46:34.805620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 15:46:34.806611: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-27 15:46:34.806631: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-27 15:46:34.806642: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-27 15:46:34.808631: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-27 15:46:35.172440: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 15:46:35.172534: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 15:46:35.172557: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 15:46:35.172578: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 15:46:35.172599: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 15:46:35.172620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 15:46:35.172639: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 15:46:35.172660: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 15:46:35.174258: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 15:46:35.176133: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 15:46:35.176212: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 15:46:35.176240: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 15:46:35.176264: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 15:46:35.176288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 15:46:35.176312: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 15:46:35.176335: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 15:46:35.176358: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 15:46:35.178546: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 15:46:35.178599: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-27 15:46:35.178615: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-27 15:46:35.178628: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-27 15:46:35.181001: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-27 15:46:47.390094: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 15:46:47.571138: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 3 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 8 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 24 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 11 Detection mask done ! Trying to reset tf kernel 4140079 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5304 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10593 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0060482025146484375 nb_pixel_total : 290242 time to create 1 rle with new method : 0.016773700714111328 length of segment : 901 time for calcul the mask position with numpy : 0.0003230571746826172 nb_pixel_total : 8169 time to create 1 rle with old method : 0.013882637023925781 length of segment : 131 time for calcul the mask position with numpy : 0.0011615753173828125 nb_pixel_total : 51079 time to create 1 rle with old method : 0.05926179885864258 length of segment : 308 time for calcul the mask position with numpy : 0.0004572868347167969 nb_pixel_total : 23280 time to create 1 rle with old method : 0.026254892349243164 length of segment : 288 time for calcul the mask position with numpy : 0.0005712509155273438 nb_pixel_total : 27117 time to create 1 rle with old method : 0.03228402137756348 length of segment : 172 time for calcul the mask position with numpy : 0.0007050037384033203 nb_pixel_total : 29274 time to create 1 rle with old method : 0.03377270698547363 length of segment : 573 time for calcul the mask position with numpy : 0.0007116794586181641 nb_pixel_total : 16620 time to create 1 rle with old method : 0.028341293334960938 length of segment : 261 time for calcul the mask position with numpy : 0.00014328956604003906 nb_pixel_total : 3731 time to create 1 rle with old method : 0.00441431999206543 length of segment : 67 time for calcul the mask position with numpy : 0.0011417865753173828 nb_pixel_total : 30589 time to create 1 rle with old method : 0.03866910934448242 length of segment : 504 time for calcul the mask position with numpy : 0.0003657341003417969 nb_pixel_total : 15954 time to create 1 rle with old method : 0.018901348114013672 length of segment : 219 time for calcul the mask position with numpy : 0.0008485317230224609 nb_pixel_total : 52334 time to create 1 rle with old method : 0.06442809104919434 length of segment : 273 time for calcul the mask position with numpy : 0.00043010711669921875 nb_pixel_total : 15918 time to create 1 rle with old method : 0.018999338150024414 length of segment : 170 time for calcul the mask position with numpy : 0.0032796859741210938 nb_pixel_total : 198677 time to create 1 rle with new method : 0.008362054824829102 length of segment : 421 time for calcul the mask position with numpy : 0.0003209114074707031 nb_pixel_total : 7324 time to create 1 rle with old method : 0.008887529373168945 length of segment : 117 time for calcul the mask position with numpy : 0.0012006759643554688 nb_pixel_total : 46179 time to create 1 rle with old method : 0.056232452392578125 length of segment : 292 time for calcul the mask position with numpy : 0.0005204677581787109 nb_pixel_total : 9463 time to create 1 rle with old method : 0.011111021041870117 length of segment : 210 time for calcul the mask position with numpy : 0.000400543212890625 nb_pixel_total : 8129 time to create 1 rle with old method : 0.012339591979980469 length of segment : 140 time for calcul the mask position with numpy : 0.0009450912475585938 nb_pixel_total : 39986 time to create 1 rle with old method : 0.04848670959472656 length of segment : 256 time for calcul the mask position with numpy : 0.0019495487213134766 nb_pixel_total : 82205 time to create 1 rle with old method : 0.09143567085266113 length of segment : 521 time for calcul the mask position with numpy : 0.002629518508911133 nb_pixel_total : 122313 time to create 1 rle with old method : 0.1624302864074707 length of segment : 630 time for calcul the mask position with numpy : 0.0023658275604248047 nb_pixel_total : 86650 time to create 1 rle with old method : 0.10448098182678223 length of segment : 355 time for calcul the mask position with numpy : 0.00017881393432617188 nb_pixel_total : 4517 time to create 1 rle with old method : 0.005759239196777344 length of segment : 94 time for calcul the mask position with numpy : 0.0004925727844238281 nb_pixel_total : 2604 time to create 1 rle with old method : 0.005723714828491211 length of segment : 83 time for calcul the mask position with numpy : 0.0004596710205078125 nb_pixel_total : 4074 time to create 1 rle with old method : 0.006687164306640625 length of segment : 167 time for calcul the mask position with numpy : 0.00013566017150878906 nb_pixel_total : 3534 time to create 1 rle with old method : 0.004498481750488281 length of segment : 77 time for calcul the mask position with numpy : 0.000797271728515625 nb_pixel_total : 44675 time to create 1 rle with old method : 0.048201560974121094 length of segment : 519 time for calcul the mask position with numpy : 0.0005631446838378906 nb_pixel_total : 29939 time to create 1 rle with old method : 0.033943891525268555 length of segment : 328 time for calcul the mask position with numpy : 0.0016298294067382812 nb_pixel_total : 99836 time to create 1 rle with old method : 0.11156415939331055 length of segment : 546 time for calcul the mask position with numpy : 0.0008170604705810547 nb_pixel_total : 42611 time to create 1 rle with old method : 0.047934532165527344 length of segment : 210 time for calcul the mask position with numpy : 0.00023317337036132812 nb_pixel_total : 13482 time to create 1 rle with old method : 0.015486955642700195 length of segment : 176 time for calcul the mask position with numpy : 0.0005407333374023438 nb_pixel_total : 40473 time to create 1 rle with old method : 0.04330706596374512 length of segment : 227 time for calcul the mask position with numpy : 6.246566772460938e-05 nb_pixel_total : 1142 time to create 1 rle with old method : 0.0014209747314453125 length of segment : 46 time for calcul the mask position with numpy : 0.0011839866638183594 nb_pixel_total : 71057 time to create 1 rle with old method : 0.08491086959838867 length of segment : 280 time for calcul the mask position with numpy : 0.003062009811401367 nb_pixel_total : 163395 time to create 1 rle with new method : 0.01003885269165039 length of segment : 431 time for calcul the mask position with numpy : 0.0003941059112548828 nb_pixel_total : 23187 time to create 1 rle with old method : 0.02715611457824707 length of segment : 171 time for calcul the mask position with numpy : 0.004805326461791992 nb_pixel_total : 219271 time to create 1 rle with new method : 0.01814746856689453 length of segment : 607 time for calcul the mask position with numpy : 0.000316619873046875 nb_pixel_total : 13507 time to create 1 rle with old method : 0.015881061553955078 length of segment : 170 time for calcul the mask position with numpy : 0.001703500747680664 nb_pixel_total : 106152 time to create 1 rle with old method : 0.12404322624206543 length of segment : 364 time for calcul the mask position with numpy : 0.00015091896057128906 nb_pixel_total : 2022 time to create 1 rle with old method : 0.002621173858642578 length of segment : 77 time for calcul the mask position with numpy : 0.003767251968383789 nb_pixel_total : 143042 time to create 1 rle with old method : 0.16197919845581055 length of segment : 362 time for calcul the mask position with numpy : 0.00726008415222168 nb_pixel_total : 192066 time to create 1 rle with new method : 0.011095046997070312 length of segment : 589 time for calcul the mask position with numpy : 0.005271434783935547 nb_pixel_total : 152614 time to create 1 rle with new method : 0.007962465286254883 length of segment : 566 time for calcul the mask position with numpy : 0.000530242919921875 nb_pixel_total : 7968 time to create 1 rle with old method : 0.0092010498046875 length of segment : 103 time for calcul the mask position with numpy : 0.0038285255432128906 nb_pixel_total : 88735 time to create 1 rle with old method : 0.11637377738952637 length of segment : 415 time for calcul the mask position with numpy : 0.002206087112426758 nb_pixel_total : 63243 time to create 1 rle with old method : 0.07831430435180664 length of segment : 244 time for calcul the mask position with numpy : 0.0009298324584960938 nb_pixel_total : 21375 time to create 1 rle with old method : 0.029523611068725586 length of segment : 610 time for calcul the mask position with numpy : 0.00041961669921875 nb_pixel_total : 12427 time to create 1 rle with old method : 0.015187501907348633 length of segment : 140 time for calcul the mask position with numpy : 0.0006513595581054688 nb_pixel_total : 16973 time to create 1 rle with old method : 0.02021026611328125 length of segment : 231 time for calcul the mask position with numpy : 0.0013134479522705078 nb_pixel_total : 39455 time to create 1 rle with old method : 0.05564737319946289 length of segment : 376 time for calcul the mask position with numpy : 0.0017709732055664062 nb_pixel_total : 44838 time to create 1 rle with old method : 0.052190542221069336 length of segment : 379 time for calcul the mask position with numpy : 0.0006644725799560547 nb_pixel_total : 12419 time to create 1 rle with old method : 0.01725912094116211 length of segment : 163 time for calcul the mask position with numpy : 0.0003917217254638672 nb_pixel_total : 5877 time to create 1 rle with old method : 0.0073833465576171875 length of segment : 100 time for calcul the mask position with numpy : 0.0016207695007324219 nb_pixel_total : 55304 time to create 1 rle with old method : 0.062424659729003906 length of segment : 307 time for calcul the mask position with numpy : 0.0008244514465332031 nb_pixel_total : 27459 time to create 1 rle with old method : 0.032872676849365234 length of segment : 162 time for calcul the mask position with numpy : 0.0009481906890869141 nb_pixel_total : 23395 time to create 1 rle with old method : 0.028809547424316406 length of segment : 219 time for calcul the mask position with numpy : 0.0011284351348876953 nb_pixel_total : 19364 time to create 1 rle with old method : 0.03225088119506836 length of segment : 258 time for calcul the mask position with numpy : 0.001889944076538086 nb_pixel_total : 37418 time to create 1 rle with old method : 0.045331478118896484 length of segment : 401 time for calcul the mask position with numpy : 0.006047248840332031 nb_pixel_total : 196507 time to create 1 rle with new method : 0.009354352951049805 length of segment : 519 time for calcul the mask position with numpy : 0.00151824951171875 nb_pixel_total : 42856 time to create 1 rle with old method : 0.04972338676452637 length of segment : 266 time for calcul the mask position with numpy : 0.0017535686492919922 nb_pixel_total : 50092 time to create 1 rle with old method : 0.05703377723693848 length of segment : 310 time for calcul the mask position with numpy : 0.0008330345153808594 nb_pixel_total : 8784 time to create 1 rle with old method : 0.010499238967895508 length of segment : 275 time for calcul the mask position with numpy : 0.0006957054138183594 nb_pixel_total : 44510 time to create 1 rle with old method : 0.0524899959564209 length of segment : 260 time for calcul the mask position with numpy : 0.003790616989135742 nb_pixel_total : 90608 time to create 1 rle with old method : 0.11897921562194824 length of segment : 351 time for calcul the mask position with numpy : 0.006309986114501953 nb_pixel_total : 170462 time to create 1 rle with new method : 0.010871171951293945 length of segment : 604 time for calcul the mask position with numpy : 0.0007843971252441406 nb_pixel_total : 20658 time to create 1 rle with old method : 0.024312973022460938 length of segment : 126 time for calcul the mask position with numpy : 0.00013971328735351562 nb_pixel_total : 2166 time to create 1 rle with old method : 0.002907276153564453 length of segment : 50 time for calcul the mask position with numpy : 0.00022935867309570312 nb_pixel_total : 3992 time to create 1 rle with old method : 0.005252838134765625 length of segment : 52 time for calcul the mask position with numpy : 0.0010678768157958984 nb_pixel_total : 24066 time to create 1 rle with old method : 0.028270483016967773 length of segment : 303 time for calcul the mask position with numpy : 0.0006117820739746094 nb_pixel_total : 10447 time to create 1 rle with old method : 0.017411231994628906 length of segment : 143 time for calcul the mask position with numpy : 0.002456188201904297 nb_pixel_total : 47665 time to create 1 rle with old method : 0.06196904182434082 length of segment : 370 time for calcul the mask position with numpy : 0.0030412673950195312 nb_pixel_total : 115357 time to create 1 rle with old method : 0.1282353401184082 length of segment : 414 time for calcul the mask position with numpy : 0.003412961959838867 nb_pixel_total : 146860 time to create 1 rle with old method : 0.1660292148590088 length of segment : 411 time for calcul the mask position with numpy : 0.00020694732666015625 nb_pixel_total : 2956 time to create 1 rle with old method : 0.003879547119140625 length of segment : 45 time for calcul the mask position with numpy : 0.005491971969604492 nb_pixel_total : 224651 time to create 1 rle with new method : 0.007565498352050781 length of segment : 605 time for calcul the mask position with numpy : 0.00038886070251464844 nb_pixel_total : 6643 time to create 1 rle with old method : 0.008365392684936523 length of segment : 156 time for calcul the mask position with numpy : 0.0018353462219238281 nb_pixel_total : 43645 time to create 1 rle with old method : 0.050780534744262695 length of segment : 328 time for calcul the mask position with numpy : 0.0010488033294677734 nb_pixel_total : 49038 time to create 1 rle with old method : 0.05863332748413086 length of segment : 197 time for calcul the mask position with numpy : 0.0003097057342529297 nb_pixel_total : 9297 time to create 1 rle with old method : 0.011439323425292969 length of segment : 116 time for calcul the mask position with numpy : 0.0009341239929199219 nb_pixel_total : 24608 time to create 1 rle with old method : 0.029987812042236328 length of segment : 272 time for calcul the mask position with numpy : 0.0009081363677978516 nb_pixel_total : 20409 time to create 1 rle with old method : 0.025168180465698242 length of segment : 239 time for calcul the mask position with numpy : 0.00508570671081543 nb_pixel_total : 206772 time to create 1 rle with new method : 0.006962299346923828 length of segment : 591 time for calcul the mask position with numpy : 0.0007352828979492188 nb_pixel_total : 20461 time to create 1 rle with old method : 0.024229049682617188 length of segment : 205 time for calcul the mask position with numpy : 0.0008907318115234375 nb_pixel_total : 52787 time to create 1 rle with old method : 0.06238889694213867 length of segment : 336 time for calcul the mask position with numpy : 0.0011997222900390625 nb_pixel_total : 21230 time to create 1 rle with old method : 0.025551557540893555 length of segment : 301 time for calcul the mask position with numpy : 0.0004105567932128906 nb_pixel_total : 12604 time to create 1 rle with old method : 0.015137195587158203 length of segment : 93 time for calcul the mask position with numpy : 0.00473332405090332 nb_pixel_total : 142266 time to create 1 rle with old method : 0.16721534729003906 length of segment : 497 time for calcul the mask position with numpy : 0.0002665519714355469 nb_pixel_total : 2862 time to create 1 rle with old method : 0.004042387008666992 length of segment : 54 time for calcul the mask position with numpy : 0.0011432170867919922 nb_pixel_total : 31692 time to create 1 rle with old method : 0.03932595252990723 length of segment : 235 time for calcul the mask position with numpy : 0.0007493495941162109 nb_pixel_total : 41680 time to create 1 rle with old method : 0.05011177062988281 length of segment : 156 time for calcul the mask position with numpy : 0.00048351287841796875 nb_pixel_total : 9907 time to create 1 rle with old method : 0.012838125228881836 length of segment : 138 time for calcul the mask position with numpy : 0.0003497600555419922 nb_pixel_total : 9746 time to create 1 rle with old method : 0.011791229248046875 length of segment : 125 time for calcul the mask position with numpy : 0.0005388259887695312 nb_pixel_total : 13995 time to create 1 rle with old method : 0.01699352264404297 length of segment : 156 time for calcul the mask position with numpy : 0.005445003509521484 nb_pixel_total : 231745 time to create 1 rle with new method : 0.007759571075439453 length of segment : 777 time for calcul the mask position with numpy : 0.0002491474151611328 nb_pixel_total : 4133 time to create 1 rle with old method : 0.005553007125854492 length of segment : 44 time for calcul the mask position with numpy : 0.0003192424774169922 nb_pixel_total : 9760 time to create 1 rle with old method : 0.013180732727050781 length of segment : 58 time for calcul the mask position with numpy : 0.0004420280456542969 nb_pixel_total : 10186 time to create 1 rle with old method : 0.012135744094848633 length of segment : 123 time for calcul the mask position with numpy : 0.004314422607421875 nb_pixel_total : 179381 time to create 1 rle with new method : 0.005705356597900391 length of segment : 629 time for calcul the mask position with numpy : 0.0015759468078613281 nb_pixel_total : 47430 time to create 1 rle with old method : 0.05615496635437012 length of segment : 253 time for calcul the mask position with numpy : 0.0012559890747070312 nb_pixel_total : 58117 time to create 1 rle with old method : 0.06927776336669922 length of segment : 267 time for calcul the mask position with numpy : 0.0019521713256835938 nb_pixel_total : 44408 time to create 1 rle with old method : 0.05470442771911621 length of segment : 372 time for calcul the mask position with numpy : 0.0019884109497070312 nb_pixel_total : 56191 time to create 1 rle with old method : 0.06436300277709961 length of segment : 525 time for calcul the mask position with numpy : 0.010378122329711914 nb_pixel_total : 352495 time to create 1 rle with new method : 0.018263578414916992 length of segment : 815 time for calcul the mask position with numpy : 0.0002770423889160156 nb_pixel_total : 4081 time to create 1 rle with old method : 0.005314350128173828 length of segment : 83 time for calcul the mask position with numpy : 0.0007700920104980469 nb_pixel_total : 13647 time to create 1 rle with old method : 0.0171053409576416 length of segment : 190 time for calcul the mask position with numpy : 0.02206873893737793 nb_pixel_total : 769220 time to create 1 rle with new method : 0.0367584228515625 length of segment : 909 time for calcul the mask position with numpy : 0.0002105236053466797 nb_pixel_total : 8917 time to create 1 rle with old method : 0.01137685775756836 length of segment : 69 time for calcul the mask position with numpy : 0.0040950775146484375 nb_pixel_total : 157477 time to create 1 rle with new method : 0.005695819854736328 length of segment : 551 time for calcul the mask position with numpy : 0.0026886463165283203 nb_pixel_total : 61229 time to create 1 rle with old method : 0.08171510696411133 length of segment : 327 time for calcul the mask position with numpy : 0.000865936279296875 nb_pixel_total : 16719 time to create 1 rle with old method : 0.020478487014770508 length of segment : 264 time for calcul the mask position with numpy : 0.0024356842041015625 nb_pixel_total : 94716 time to create 1 rle with old method : 0.1325392723083496 length of segment : 374 time for calcul the mask position with numpy : 0.0019652843475341797 nb_pixel_total : 31210 time to create 1 rle with old method : 0.0383458137512207 length of segment : 285 time for calcul the mask position with numpy : 0.0006473064422607422 nb_pixel_total : 11874 time to create 1 rle with old method : 0.01466512680053711 length of segment : 158 time for calcul the mask position with numpy : 0.0035839080810546875 nb_pixel_total : 120627 time to create 1 rle with old method : 0.14256882667541504 length of segment : 641 time for calcul the mask position with numpy : 0.0051653385162353516 nb_pixel_total : 161875 time to create 1 rle with new method : 0.007966756820678711 length of segment : 684 time for calcul the mask position with numpy : 0.002016782760620117 nb_pixel_total : 36718 time to create 1 rle with old method : 0.043421268463134766 length of segment : 395 time for calcul the mask position with numpy : 0.002360105514526367 nb_pixel_total : 51534 time to create 1 rle with old method : 0.07342886924743652 length of segment : 518 time for calcul the mask position with numpy : 0.0076253414154052734 nb_pixel_total : 308373 time to create 1 rle with new method : 0.011811017990112305 length of segment : 896 time for calcul the mask position with numpy : 0.0022194385528564453 nb_pixel_total : 53470 time to create 1 rle with old method : 0.062335968017578125 length of segment : 315 time for calcul the mask position with numpy : 0.00024247169494628906 nb_pixel_total : 2213 time to create 1 rle with old method : 0.0040149688720703125 length of segment : 74 time for calcul the mask position with numpy : 0.00032830238342285156 nb_pixel_total : 10698 time to create 1 rle with old method : 0.01776123046875 length of segment : 192 time for calcul the mask position with numpy : 0.0022950172424316406 nb_pixel_total : 109013 time to create 1 rle with old method : 0.12800073623657227 length of segment : 542 time for calcul the mask position with numpy : 0.0006785392761230469 nb_pixel_total : 23875 time to create 1 rle with old method : 0.028589487075805664 length of segment : 143 time for calcul the mask position with numpy : 0.0010311603546142578 nb_pixel_total : 30046 time to create 1 rle with old method : 0.036444902420043945 length of segment : 480 time for calcul the mask position with numpy : 0.0001895427703857422 nb_pixel_total : 1722 time to create 1 rle with old method : 0.0021610260009765625 length of segment : 81 time for calcul the mask position with numpy : 0.0518345832824707 nb_pixel_total : 1621374 time to create 1 rle with new method : 0.1726064682006836 length of segment : 2566 time for calcul the mask position with numpy : 0.009388208389282227 nb_pixel_total : 300297 time to create 1 rle with new method : 0.014995098114013672 length of segment : 767 time for calcul the mask position with numpy : 0.003504037857055664 nb_pixel_total : 237064 time to create 1 rle with new method : 0.008143186569213867 length of segment : 455 time for calcul the mask position with numpy : 0.005467414855957031 nb_pixel_total : 272575 time to create 1 rle with new method : 0.009978055953979492 length of segment : 659 time for calcul the mask position with numpy : 0.0006773471832275391 nb_pixel_total : 13702 time to create 1 rle with old method : 0.01622629165649414 length of segment : 199 time for calcul the mask position with numpy : 0.0005717277526855469 nb_pixel_total : 16772 time to create 1 rle with old method : 0.02045607566833496 length of segment : 326 time for calcul the mask position with numpy : 0.00034332275390625 nb_pixel_total : 20454 time to create 1 rle with old method : 0.02490830421447754 length of segment : 165 time for calcul the mask position with numpy : 0.00027561187744140625 nb_pixel_total : 3350 time to create 1 rle with old method : 0.003736734390258789 length of segment : 149 time for calcul the mask position with numpy : 0.0015175342559814453 nb_pixel_total : 71669 time to create 1 rle with old method : 0.08474588394165039 length of segment : 496 time for calcul the mask position with numpy : 0.0002655982971191406 nb_pixel_total : 10658 time to create 1 rle with old method : 0.013257741928100586 length of segment : 127 time for calcul the mask position with numpy : 0.0005633831024169922 nb_pixel_total : 11343 time to create 1 rle with old method : 0.016629934310913086 length of segment : 210 time for calcul the mask position with numpy : 0.004937887191772461 nb_pixel_total : 93909 time to create 1 rle with old method : 0.10744237899780273 length of segment : 968 time for calcul the mask position with numpy : 0.005563974380493164 nb_pixel_total : 302495 time to create 1 rle with new method : 0.01398777961730957 length of segment : 718 time for calcul the mask position with numpy : 0.0008149147033691406 nb_pixel_total : 17686 time to create 1 rle with old method : 0.021347761154174805 length of segment : 300 time for calcul the mask position with numpy : 0.003782510757446289 nb_pixel_total : 239708 time to create 1 rle with new method : 0.0102996826171875 length of segment : 455 time for calcul the mask position with numpy : 0.0004067420959472656 nb_pixel_total : 4760 time to create 1 rle with old method : 0.006192207336425781 length of segment : 93 time for calcul the mask position with numpy : 0.00028324127197265625 nb_pixel_total : 4565 time to create 1 rle with old method : 0.006055116653442383 length of segment : 120 time for calcul the mask position with numpy : 0.045789241790771484 nb_pixel_total : 1876243 time to create 1 rle with new method : 0.8315486907958984 length of segment : 3155 time for calcul the mask position with numpy : 0.000324249267578125 nb_pixel_total : 10149 time to create 1 rle with old method : 0.011970281600952148 length of segment : 134 time for calcul the mask position with numpy : 0.005393028259277344 nb_pixel_total : 366658 time to create 1 rle with new method : 0.013935327529907227 length of segment : 973 time for calcul the mask position with numpy : 0.023495197296142578 nb_pixel_total : 653937 time to create 1 rle with new method : 0.056725263595581055 length of segment : 2140 time for calcul the mask position with numpy : 0.004922151565551758 nb_pixel_total : 391591 time to create 1 rle with new method : 0.012351512908935547 length of segment : 979 time for calcul the mask position with numpy : 0.0018055438995361328 nb_pixel_total : 123061 time to create 1 rle with old method : 0.13654613494873047 length of segment : 584 time for calcul the mask position with numpy : 0.004790306091308594 nb_pixel_total : 431945 time to create 1 rle with new method : 0.009322166442871094 length of segment : 996 time for calcul the mask position with numpy : 0.002801179885864258 nb_pixel_total : 207886 time to create 1 rle with new method : 0.0071675777435302734 length of segment : 909 time for calcul the mask position with numpy : 0.013154745101928711 nb_pixel_total : 751430 time to create 1 rle with new method : 0.029686450958251953 length of segment : 1350 time spent for convertir_results : 15.547894954681396 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 161 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 58333 save missing photos in datou_result : time spend for datou_step_exec : 77.24056768417358 time spend to save output : 3.3242223262786865 total time spend for step 1 : 80.56479001045227 step2:crop_condition Thu Feb 27 15:47:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 161 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 13 About to insert : list_path_to_insert length 13 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 39 About to insert : list_path_to_insert length 39 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339803007, 1339803004, 1339802998, 1339802988, 1339802842, 1339802837, 1339802834, 1339768174, 1339768171, 1339768168] Looping around the photos to save general results len do output : 79 /-3688685166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685162Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685185Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685186Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685192Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685197Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685191Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685194Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685277Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688766759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688766758Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685247Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685271Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685279Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685293Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685184Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688766752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685164Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685173Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685156Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688766754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685179Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685182Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685187Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685189Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685193Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685195Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685196Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688766757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685220Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685243Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685239Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685248Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685242Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685249Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685270Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685268Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685264Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688766762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685241Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685250Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3688685299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803007', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803004', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802998', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802988', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802842', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802837', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802834', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768174', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768171', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768168', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 247 time used for this insertion : 0.032372474670410156 save_final save missing photos in datou_result : time spend for datou_step_exec : 35.6508309841156 time spend to save output : 0.03434300422668457 total time spend for step 2 : 35.685173988342285 step3:thcl Thu Feb 27 15:48:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.012159585952758789 time to convert the images to numpy array : 0.0980691909790039 time to import caffe and check if the image exist : 0.01858043670654297 time to convert the images to numpy array : 0.10529637336730957 time to import caffe and check if the image exist : 0.010080099105834961 time to convert the images to numpy array : 0.12224197387695312 time to import caffe and check if the image exist : 0.015244007110595703 time to convert the images to numpy array : 0.12160873413085938 time to import caffe and check if the image exist : 0.01707744598388672 time to convert the images to numpy array : 0.11997652053833008 time to import caffe and check if the image exist : 0.012330293655395508 time to convert the images to numpy array : 0.1283566951751709 time to import caffe and check if the image exist : 0.012630701065063477 time to convert the images to numpy array : 0.13719654083251953 time to import caffe and check if the image exist : 0.011133909225463867 time to convert the images to numpy array : 0.1566023826599121 time to import caffe and check if the image exist : 0.013395547866821289 time to convert the images to numpy array : 0.156174898147583 time to import caffe and check if the image exist : 0.018208026885986328 time to convert the images to numpy array : 0.16338229179382324 total time to convert the images to numpy array : 0.49253082275390625 list photo_ids error: [] list photo_ids correct : [-3688685194, -3688685203, -3688685216, -3688685215, -3688685223, -3688685235, -3688685236, -3688685234, -3688685166, -3688685162, -3688685185, -3688685186, -3688685205, -3688685192, -3688685197, -3688685191, -3688685184, -3688685272, -3688766752, -3688685164, -3688685173, -3688685156, -3688685169, -3688685172, -3688685256, -3688685277, -3688766759, -3688685167, -3688685160, -3688685204, -3688685201, -3688685210, -3688685209, -3688685213, -3688685206, -3688685195, -3688685214, -3688685196, -3688685207, -3688766757, -3688685222, -3688685220, -3688685243, -3688685239, -3688685248, -3688685242, -3688685249, -3688685258, -3688685261, -3688685270, -3688685268, -3688685264, -3688685265, -3688685260, -3688685289, -3688766762, -3688766754, -3688685171, -3688685179, -3688685182, -3688685187, -3688685189, -3688685211, -3688685193, -3688685200, -3688685233, -3688766758, -3688685247, -3688685271, -3688685287, -3688685279, -3688685293, -3688685294, -3688685155, -3688685159, -3688685241, -3688685250, -3688685298, -3688685299] number of photos to traite : 79 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10595 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.8781137466430664 time used to do the prediction : 0.1966240406036377 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10593 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.9374008178710938 time used to do the prediction : 0.1438896656036377 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.8835067749023438e-05 save missing photos in datou_result : time spend for datou_step_exec : 9.364941120147705 time spend to save output : 0.0005609989166259766 total time spend for step 3 : 9.365502119064331 step4:merge_mask_thcl_custom Thu Feb 27 15:48:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 161 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1339803007, 1339803004, 1339802998, 1339802988, 1339802842, 1339802837, 1339802834, 1339768174, 1339768171, 1339768168] Looping around the photos to save general results len do output : 10 /1339803007Didn't retrieve data .Didn't retrieve data . /1339803004Didn't retrieve data .Didn't retrieve data . /1339802998Didn't retrieve data .Didn't retrieve data . /1339802988Didn't retrieve data .Didn't retrieve data . /1339802842Didn't retrieve data .Didn't retrieve data . /1339802837Didn't retrieve data .Didn't retrieve data . /1339802834Didn't retrieve data .Didn't retrieve data . /1339768174Didn't retrieve data .Didn't retrieve data . /1339768171Didn't retrieve data .Didn't retrieve data . /1339768168Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803007', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803004', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802998', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802988', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802842', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802837', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802834', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768174', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768171', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768168', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.09077882766723633 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.0625457763671875 time spend to save output : 0.09128737449645996 total time spend for step 4 : 0.15383315086364746 step5:rle_unique_nms_with_priority Thu Feb 27 15:48:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 24 nb_hashtags : 7 time to prepare the origin masks : 13.959630489349365 time for calcul the mask position with numpy : 0.4414348602294922 nb_pixel_total : 5963757 time to create 1 rle with new method : 1.2344906330108643 time for calcul the mask position with numpy : 0.021387100219726562 nb_pixel_total : 5642 time to create 1 rle with old method : 0.006552696228027344 time for calcul the mask position with numpy : 0.021442413330078125 nb_pixel_total : 2732 time to create 1 rle with old method : 0.0032498836517333984 time for calcul the mask position with numpy : 0.02156519889831543 nb_pixel_total : 52473 time to create 1 rle with old method : 0.055875301361083984 time for calcul the mask position with numpy : 0.02197575569152832 nb_pixel_total : 46002 time to create 1 rle with old method : 0.04975390434265137 time for calcul the mask position with numpy : 0.021459579467773438 nb_pixel_total : 36724 time to create 1 rle with old method : 0.039472341537475586 time for calcul the mask position with numpy : 0.0216367244720459 nb_pixel_total : 122169 time to create 1 rle with old method : 0.13610625267028809 time for calcul the mask position with numpy : 0.021991968154907227 nb_pixel_total : 4496 time to create 1 rle with old method : 0.007209062576293945 time for calcul the mask position with numpy : 0.024919986724853516 nb_pixel_total : 86566 time to create 1 rle with old method : 0.09697079658508301 time for calcul the mask position with numpy : 0.021124601364135742 nb_pixel_total : 23257 time to create 1 rle with old method : 0.02510356903076172 time for calcul the mask position with numpy : 0.020671367645263672 nb_pixel_total : 9363 time to create 1 rle with old method : 0.010778188705444336 time for calcul the mask position with numpy : 0.021442413330078125 nb_pixel_total : 26916 time to create 1 rle with old method : 0.03171825408935547 time for calcul the mask position with numpy : 0.02235889434814453 nb_pixel_total : 8057 time to create 1 rle with old method : 0.009435176849365234 time for calcul the mask position with numpy : 0.022747039794921875 nb_pixel_total : 3712 time to create 1 rle with old method : 0.004184246063232422 time for calcul the mask position with numpy : 0.022521257400512695 nb_pixel_total : 7278 time to create 1 rle with old method : 0.008595705032348633 time for calcul the mask position with numpy : 0.023279905319213867 nb_pixel_total : 30572 time to create 1 rle with old method : 0.03494548797607422 time for calcul the mask position with numpy : 0.021906614303588867 nb_pixel_total : 15868 time to create 1 rle with old method : 0.01854085922241211 time for calcul the mask position with numpy : 0.021256446838378906 nb_pixel_total : 15925 time to create 1 rle with old method : 0.01794743537902832 time for calcul the mask position with numpy : 0.02117466926574707 nb_pixel_total : 11359 time to create 1 rle with old method : 0.013227701187133789 time for calcul the mask position with numpy : 0.02136826515197754 nb_pixel_total : 291 time to create 1 rle with old method : 0.00048351287841796875 time for calcul the mask position with numpy : 0.02146005630493164 nb_pixel_total : 29220 time to create 1 rle with old method : 0.03434562683105469 time for calcul the mask position with numpy : 0.021746158599853516 nb_pixel_total : 8100 time to create 1 rle with old method : 0.009351015090942383 time for calcul the mask position with numpy : 0.0348200798034668 nb_pixel_total : 289543 time to create 1 rle with new method : 0.41722679138183594 time for calcul the mask position with numpy : 0.03551435470581055 nb_pixel_total : 52053 time to create 1 rle with old method : 0.07415533065795898 time for calcul the mask position with numpy : 0.03135395050048828 nb_pixel_total : 198165 time to create 1 rle with new method : 0.4572029113769531 create new chi : 3.881211757659912 time to delete rle : 0.018642425537109375 batch 1 Loaded 25 chid ids of type : 3726 Number RLEs to save : 15788 TO DO : save crop sub photo not yet done ! save time : 0.8790910243988037 nb_obj : 5 nb_hashtags : 2 time to prepare the origin masks : 2.8260345458984375 time for calcul the mask position with numpy : 0.4793682098388672 nb_pixel_total : 6826326 time to create 1 rle with new method : 0.7616078853607178 time for calcul the mask position with numpy : 0.034482717514038086 nb_pixel_total : 99715 time to create 1 rle with old method : 0.11139822006225586 time for calcul the mask position with numpy : 0.020207881927490234 nb_pixel_total : 42489 time to create 1 rle with old method : 0.04541015625 time for calcul the mask position with numpy : 0.02018451690673828 nb_pixel_total : 29901 time to create 1 rle with old method : 0.03332662582397461 time for calcul the mask position with numpy : 0.020987272262573242 nb_pixel_total : 48293 time to create 1 rle with old method : 0.05084967613220215 time for calcul the mask position with numpy : 0.02006697654724121 nb_pixel_total : 3516 time to create 1 rle with old method : 0.0036857128143310547 create new chi : 1.6397521495819092 time to delete rle : 0.0005044937133789062 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 5274 TO DO : save crop sub photo not yet done ! save time : 0.3301401138305664 nb_obj : 3 nb_hashtags : 2 time to prepare the origin masks : 1.8208532333374023 time for calcul the mask position with numpy : 0.4966554641723633 nb_pixel_total : 6995376 time to create 1 rle with new method : 0.634784460067749 time for calcul the mask position with numpy : 0.02081775665283203 nb_pixel_total : 1129 time to create 1 rle with old method : 0.0013065338134765625 time for calcul the mask position with numpy : 0.0216367244720459 nb_pixel_total : 40321 time to create 1 rle with old method : 0.04574918746948242 time for calcul the mask position with numpy : 0.021473407745361328 nb_pixel_total : 13414 time to create 1 rle with old method : 0.014764785766601562 create new chi : 1.2942147254943848 time to delete rle : 0.00030732154846191406 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 3060 TO DO : save crop sub photo not yet done ! save time : 0.21779847145080566 nb_obj : 7 nb_hashtags : 6 time to prepare the origin masks : 3.990973949432373 time for calcul the mask position with numpy : 0.5007495880126953 nb_pixel_total : 6454038 time to create 1 rle with new method : 0.6805157661437988 time for calcul the mask position with numpy : 0.024415254592895508 nb_pixel_total : 104829 time to create 1 rle with old method : 0.11284804344177246 time for calcul the mask position with numpy : 0.020830392837524414 nb_pixel_total : 23143 time to create 1 rle with old method : 0.025130033493041992 time for calcul the mask position with numpy : 0.020272016525268555 nb_pixel_total : 2006 time to create 1 rle with old method : 0.002162456512451172 time for calcul the mask position with numpy : 0.02156662940979004 nb_pixel_total : 218896 time to create 1 rle with new method : 0.6008694171905518 time for calcul the mask position with numpy : 0.020495891571044922 nb_pixel_total : 163065 time to create 1 rle with new method : 0.5141959190368652 time for calcul the mask position with numpy : 0.021620512008666992 nb_pixel_total : 13469 time to create 1 rle with old method : 0.015047073364257812 time for calcul the mask position with numpy : 0.02263331413269043 nb_pixel_total : 70794 time to create 1 rle with old method : 0.0768125057220459 create new chi : 2.7603559494018555 time to delete rle : 0.0009329319000244141 batch 1 Loaded 8 chid ids of type : 3726 Number RLEs to save : 6283 TO DO : save crop sub photo not yet done ! save time : 0.3641653060913086 nb_obj : 23 nb_hashtags : 8 time to prepare the origin masks : 12.558576822280884 time for calcul the mask position with numpy : 0.4046914577484131 nb_pixel_total : 5808625 time to create 1 rle with new method : 0.6552608013153076 time for calcul the mask position with numpy : 0.03870129585266113 nb_pixel_total : 142835 time to create 1 rle with old method : 0.2116539478302002 time for calcul the mask position with numpy : 0.034317731857299805 nb_pixel_total : 5715 time to create 1 rle with old method : 0.006604433059692383 time for calcul the mask position with numpy : 0.034497737884521484 nb_pixel_total : 62944 time to create 1 rle with old method : 0.06783127784729004 time for calcul the mask position with numpy : 0.03426718711853027 nb_pixel_total : 88643 time to create 1 rle with old method : 0.1175088882446289 time for calcul the mask position with numpy : 0.03566408157348633 nb_pixel_total : 23351 time to create 1 rle with old method : 0.029484987258911133 time for calcul the mask position with numpy : 0.038046836853027344 nb_pixel_total : 3708 time to create 1 rle with old method : 0.005444765090942383 time for calcul the mask position with numpy : 0.0643608570098877 nb_pixel_total : 5526 time to create 1 rle with old method : 0.008444070816040039 time for calcul the mask position with numpy : 0.03783607482910156 nb_pixel_total : 49874 time to create 1 rle with old method : 0.05694270133972168 time for calcul the mask position with numpy : 0.03665494918823242 nb_pixel_total : 196297 time to create 1 rle with new method : 0.6397209167480469 time for calcul the mask position with numpy : 0.022335290908813477 nb_pixel_total : 37317 time to create 1 rle with old method : 0.04680967330932617 time for calcul the mask position with numpy : 0.03444075584411621 nb_pixel_total : 42623 time to create 1 rle with old method : 0.0466914176940918 time for calcul the mask position with numpy : 0.03450179100036621 nb_pixel_total : 27411 time to create 1 rle with old method : 0.0312952995300293 time for calcul the mask position with numpy : 0.03677487373352051 nb_pixel_total : 19283 time to create 1 rle with old method : 0.02239060401916504 time for calcul the mask position with numpy : 0.03401303291320801 nb_pixel_total : 12368 time to create 1 rle with old method : 0.01386404037475586 time for calcul the mask position with numpy : 0.03242754936218262 nb_pixel_total : 171830 time to create 1 rle with new method : 1.0392074584960938 time for calcul the mask position with numpy : 0.03105020523071289 nb_pixel_total : 5846 time to create 1 rle with old method : 0.0064465999603271484 time for calcul the mask position with numpy : 0.031241655349731445 nb_pixel_total : 16923 time to create 1 rle with old method : 0.018130064010620117 time for calcul the mask position with numpy : 0.03115391731262207 nb_pixel_total : 12413 time to create 1 rle with old method : 0.013768911361694336 time for calcul the mask position with numpy : 0.03126192092895508 nb_pixel_total : 152451 time to create 1 rle with new method : 0.6602458953857422 time for calcul the mask position with numpy : 0.030539274215698242 nb_pixel_total : 21266 time to create 1 rle with old method : 0.024522781372070312 time for calcul the mask position with numpy : 0.03020644187927246 nb_pixel_total : 90475 time to create 1 rle with old method : 0.09823131561279297 time for calcul the mask position with numpy : 0.026193618774414062 nb_pixel_total : 7954 time to create 1 rle with old method : 0.008707761764526367 time for calcul the mask position with numpy : 0.02405571937561035 nb_pixel_total : 44562 time to create 1 rle with old method : 0.052568674087524414 create new chi : 5.1756227016448975 time to delete rle : 0.0013942718505859375 batch 1 Loaded 24 chid ids of type : 3726 Number RLEs to save : 15979 TO DO : save crop sub photo not yet done ! save time : 2.2226908206939697 nb_obj : 21 nb_hashtags : 6 time to prepare the origin masks : 13.317102432250977 time for calcul the mask position with numpy : 0.8656363487243652 nb_pixel_total : 5910494 time to create 1 rle with new method : 1.2606861591339111 time for calcul the mask position with numpy : 0.04188942909240723 nb_pixel_total : 47472 time to create 1 rle with old method : 0.052996158599853516 time for calcul the mask position with numpy : 0.04222559928894043 nb_pixel_total : 146782 time to create 1 rle with old method : 0.16244864463806152 time for calcul the mask position with numpy : 0.042047739028930664 nb_pixel_total : 20414 time to create 1 rle with old method : 0.022060871124267578 time for calcul the mask position with numpy : 0.038968563079833984 nb_pixel_total : 9557 time to create 1 rle with old method : 0.010355949401855469 time for calcul the mask position with numpy : 0.04218435287475586 nb_pixel_total : 206617 time to create 1 rle with new method : 1.5223090648651123 time for calcul the mask position with numpy : 0.04297018051147461 nb_pixel_total : 20579 time to create 1 rle with old method : 0.023319482803344727 time for calcul the mask position with numpy : 0.04002642631530762 nb_pixel_total : 24517 time to create 1 rle with old method : 0.0269927978515625 time for calcul the mask position with numpy : 0.041202545166015625 nb_pixel_total : 170234 time to create 1 rle with new method : 1.0598785877227783 time for calcul the mask position with numpy : 0.046869754791259766 nb_pixel_total : 9275 time to create 1 rle with old method : 0.010900020599365234 time for calcul the mask position with numpy : 0.04457497596740723 nb_pixel_total : 43425 time to create 1 rle with old method : 0.04700732231140137 time for calcul the mask position with numpy : 0.04065752029418945 nb_pixel_total : 6625 time to create 1 rle with old method : 0.007805585861206055 time for calcul the mask position with numpy : 0.04182600975036621 nb_pixel_total : 527 time to create 1 rle with old method : 0.0015151500701904297 time for calcul the mask position with numpy : 0.04700946807861328 nb_pixel_total : 226436 time to create 1 rle with new method : 0.9990484714508057 time for calcul the mask position with numpy : 0.041701316833496094 nb_pixel_total : 2918 time to create 1 rle with old method : 0.0032694339752197266 time for calcul the mask position with numpy : 0.03988361358642578 nb_pixel_total : 114877 time to create 1 rle with old method : 0.12417459487915039 time for calcul the mask position with numpy : 0.040346622467041016 nb_pixel_total : 229 time to create 1 rle with old method : 0.0003948211669921875 time for calcul the mask position with numpy : 0.03907060623168945 nb_pixel_total : 10417 time to create 1 rle with old method : 0.011442899703979492 time for calcul the mask position with numpy : 0.03819012641906738 nb_pixel_total : 23890 time to create 1 rle with old method : 0.02509593963623047 time for calcul the mask position with numpy : 0.03870368003845215 nb_pixel_total : 3963 time to create 1 rle with old method : 0.0043964385986328125 time for calcul the mask position with numpy : 0.04287409782409668 nb_pixel_total : 2143 time to create 1 rle with old method : 0.0026350021362304688 time for calcul the mask position with numpy : 0.03667879104614258 nb_pixel_total : 48849 time to create 1 rle with old method : 0.05446171760559082 create new chi : 7.301928997039795 time to delete rle : 0.0020754337310791016 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 13143 TO DO : save crop sub photo not yet done ! save time : 0.7875089645385742 nb_obj : 19 nb_hashtags : 7 time to prepare the origin masks : 16.259031534194946 time for calcul the mask position with numpy : 0.6163575649261475 nb_pixel_total : 5768581 time to create 1 rle with new method : 0.6024265289306641 time for calcul the mask position with numpy : 0.038848161697387695 nb_pixel_total : 179013 time to create 1 rle with new method : 0.6770017147064209 time for calcul the mask position with numpy : 0.03631877899169922 nb_pixel_total : 9727 time to create 1 rle with old method : 0.011337518692016602 time for calcul the mask position with numpy : 0.0355224609375 nb_pixel_total : 13947 time to create 1 rle with old method : 0.016365766525268555 time for calcul the mask position with numpy : 0.04142642021179199 nb_pixel_total : 348510 time to create 1 rle with new method : 0.7941646575927734 time for calcul the mask position with numpy : 0.03529667854309082 nb_pixel_total : 57473 time to create 1 rle with old method : 0.08049273490905762 time for calcul the mask position with numpy : 0.03717923164367676 nb_pixel_total : 142153 time to create 1 rle with old method : 0.17244267463684082 time for calcul the mask position with numpy : 0.035448551177978516 nb_pixel_total : 21187 time to create 1 rle with old method : 0.02916550636291504 time for calcul the mask position with numpy : 0.036821842193603516 nb_pixel_total : 237664 time to create 1 rle with new method : 1.0691518783569336 time for calcul the mask position with numpy : 0.03515744209289551 nb_pixel_total : 44505 time to create 1 rle with old method : 0.054717302322387695 time for calcul the mask position with numpy : 0.03437948226928711 nb_pixel_total : 10144 time to create 1 rle with old method : 0.011415719985961914 time for calcul the mask position with numpy : 0.03527188301086426 nb_pixel_total : 12544 time to create 1 rle with old method : 0.016010522842407227 time for calcul the mask position with numpy : 0.03833127021789551 nb_pixel_total : 57930 time to create 1 rle with old method : 0.07185745239257812 time for calcul the mask position with numpy : 0.03865480422973633 nb_pixel_total : 9707 time to create 1 rle with old method : 0.011517524719238281 time for calcul the mask position with numpy : 0.0445103645324707 nb_pixel_total : 4083 time to create 1 rle with old method : 0.004717826843261719 time for calcul the mask position with numpy : 0.03929424285888672 nb_pixel_total : 47275 time to create 1 rle with old method : 0.05588865280151367 time for calcul the mask position with numpy : 0.03965044021606445 nb_pixel_total : 9867 time to create 1 rle with old method : 0.011445283889770508 time for calcul the mask position with numpy : 0.03230929374694824 nb_pixel_total : 31525 time to create 1 rle with old method : 0.03770303726196289 time for calcul the mask position with numpy : 0.02620720863342285 nb_pixel_total : 2853 time to create 1 rle with old method : 0.0034019947052001953 time for calcul the mask position with numpy : 0.035759925842285156 nb_pixel_total : 41552 time to create 1 rle with old method : 0.0538022518157959 create new chi : 5.220785856246948 time to delete rle : 0.0023212432861328125 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 12781 TO DO : save crop sub photo not yet done ! save time : 0.7498335838317871 nb_obj : 21 nb_hashtags : 4 time to prepare the origin masks : 17.658909797668457 time for calcul the mask position with numpy : 0.2382948398590088 nb_pixel_total : 5049075 time to create 1 rle with new method : 0.6253945827484131 time for calcul the mask position with numpy : 0.03473377227783203 nb_pixel_total : 2103 time to create 1 rle with old method : 0.002655506134033203 time for calcul the mask position with numpy : 0.03632664680480957 nb_pixel_total : 174947 time to create 1 rle with new method : 0.6991329193115234 time for calcul the mask position with numpy : 0.040235042572021484 nb_pixel_total : 29978 time to create 1 rle with old method : 0.03332948684692383 time for calcul the mask position with numpy : 0.036887168884277344 nb_pixel_total : 61019 time to create 1 rle with old method : 0.06753730773925781 time for calcul the mask position with numpy : 0.03647589683532715 nb_pixel_total : 2197 time to create 1 rle with old method : 0.0025289058685302734 time for calcul the mask position with numpy : 0.038019418716430664 nb_pixel_total : 308170 time to create 1 rle with new method : 0.8616795539855957 time for calcul the mask position with numpy : 0.03682279586791992 nb_pixel_total : 51310 time to create 1 rle with old method : 0.05849266052246094 time for calcul the mask position with numpy : 0.03511357307434082 nb_pixel_total : 10491 time to create 1 rle with old method : 0.011641263961791992 time for calcul the mask position with numpy : 0.03763175010681152 nb_pixel_total : 141832 time to create 1 rle with old method : 0.1567533016204834 time for calcul the mask position with numpy : 0.03557300567626953 nb_pixel_total : 11820 time to create 1 rle with old method : 0.014071464538574219 time for calcul the mask position with numpy : 0.03587222099304199 nb_pixel_total : 16655 time to create 1 rle with old method : 0.018455028533935547 time for calcul the mask position with numpy : 0.03544330596923828 nb_pixel_total : 23855 time to create 1 rle with old method : 0.026186466217041016 time for calcul the mask position with numpy : 0.038819313049316406 nb_pixel_total : 7679 time to create 1 rle with old method : 0.009140729904174805 time for calcul the mask position with numpy : 0.039424896240234375 nb_pixel_total : 157308 time to create 1 rle with new method : 0.8884167671203613 time for calcul the mask position with numpy : 0.044003963470458984 nb_pixel_total : 768775 time to create 1 rle with new method : 0.8247356414794922 time for calcul the mask position with numpy : 0.037384748458862305 nb_pixel_total : 13541 time to create 1 rle with old method : 0.015063047409057617 time for calcul the mask position with numpy : 0.034781455993652344 nb_pixel_total : 4031 time to create 1 rle with old method : 0.004728555679321289 time for calcul the mask position with numpy : 0.03525543212890625 nb_pixel_total : 94456 time to create 1 rle with old method : 0.10633397102355957 time for calcul the mask position with numpy : 0.03475141525268555 nb_pixel_total : 31087 time to create 1 rle with old method : 0.03455185890197754 time for calcul the mask position with numpy : 0.035514116287231445 nb_pixel_total : 53305 time to create 1 rle with old method : 0.0591588020324707 time for calcul the mask position with numpy : 0.040112972259521484 nb_pixel_total : 36606 time to create 1 rle with old method : 0.04696464538574219 create new chi : 5.733159065246582 time to delete rle : 0.0024111270904541016 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 16691 TO DO : save crop sub photo not yet done ! save time : 0.9883406162261963 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 11.631550073623657 time for calcul the mask position with numpy : 0.384763240814209 nb_pixel_total : 4495837 time to create 1 rle with new method : 0.8624837398529053 time for calcul the mask position with numpy : 0.039548635482788086 nb_pixel_total : 5935 time to create 1 rle with old method : 0.007291316986083984 time for calcul the mask position with numpy : 0.04456782341003418 nb_pixel_total : 301795 time to create 1 rle with new method : 0.8156290054321289 time for calcul the mask position with numpy : 0.043320655822753906 nb_pixel_total : 17675 time to create 1 rle with old method : 0.019723176956176758 time for calcul the mask position with numpy : 0.03669166564941406 nb_pixel_total : 4133 time to create 1 rle with old method : 0.005272388458251953 time for calcul the mask position with numpy : 0.035576820373535156 nb_pixel_total : 10590 time to create 1 rle with old method : 0.011900186538696289 time for calcul the mask position with numpy : 0.0354766845703125 nb_pixel_total : 13657 time to create 1 rle with old method : 0.015057563781738281 time for calcul the mask position with numpy : 0.028697967529296875 nb_pixel_total : 16136 time to create 1 rle with old method : 0.017889022827148438 time for calcul the mask position with numpy : 0.03618216514587402 nb_pixel_total : 229661 time to create 1 rle with new method : 0.3459336757659912 time for calcul the mask position with numpy : 0.026408910751342773 nb_pixel_total : 3349 time to create 1 rle with old method : 0.00385284423828125 time for calcul the mask position with numpy : 0.029020309448242188 nb_pixel_total : 13754 time to create 1 rle with old method : 0.026666879653930664 time for calcul the mask position with numpy : 0.030481576919555664 nb_pixel_total : 16656 time to create 1 rle with old method : 0.01971268653869629 time for calcul the mask position with numpy : 0.025285959243774414 nb_pixel_total : 299287 time to create 1 rle with new method : 0.3950793743133545 time for calcul the mask position with numpy : 0.05748939514160156 nb_pixel_total : 1620060 time to create 1 rle with new method : 0.4706559181213379 time for calcul the mask position with numpy : 0.043570756912231445 nb_pixel_total : 1715 time to create 1 rle with old method : 0.002010345458984375 create new chi : 4.04918909072876 time to delete rle : 0.0013856887817382812 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 15041 TO DO : save crop sub photo not yet done ! save time : 0.8601102828979492 nb_obj : 14 nb_hashtags : 5 time to prepare the origin masks : 12.494865894317627 time for calcul the mask position with numpy : 0.17599010467529297 nb_pixel_total : 3353603 time to create 1 rle with new method : 1.3184120655059814 time for calcul the mask position with numpy : 0.03980278968811035 nb_pixel_total : 1149 time to create 1 rle with old method : 0.002328157424926758 time for calcul the mask position with numpy : 0.03900742530822754 nb_pixel_total : 348456 time to create 1 rle with new method : 0.5329205989837646 time for calcul the mask position with numpy : 0.031455278396606445 nb_pixel_total : 50999 time to create 1 rle with old method : 0.06001687049865723 time for calcul the mask position with numpy : 0.04150843620300293 nb_pixel_total : 69828 time to create 1 rle with old method : 0.07738852500915527 time for calcul the mask position with numpy : 0.03873038291931152 nb_pixel_total : 9768 time to create 1 rle with old method : 0.011185407638549805 time for calcul the mask position with numpy : 0.043747663497924805 nb_pixel_total : 379453 time to create 1 rle with new method : 0.7331750392913818 time for calcul the mask position with numpy : 0.03996992111206055 nb_pixel_total : 195359 time to create 1 rle with new method : 0.5357465744018555 time for calcul the mask position with numpy : 0.0362393856048584 nb_pixel_total : 140762 time to create 1 rle with old method : 0.15515708923339844 time for calcul the mask position with numpy : 0.0347292423248291 nb_pixel_total : 207722 time to create 1 rle with new method : 0.6732821464538574 time for calcul the mask position with numpy : 0.052292585372924805 nb_pixel_total : 285833 time to create 1 rle with new method : 0.5158448219299316 time for calcul the mask position with numpy : 0.09803557395935059 nb_pixel_total : 1875036 time to create 1 rle with new method : 0.9058589935302734 time for calcul the mask position with numpy : 0.05205726623535156 nb_pixel_total : 4535 time to create 1 rle with old method : 0.0058193206787109375 time for calcul the mask position with numpy : 0.05314469337463379 nb_pixel_total : 4682 time to create 1 rle with old method : 0.005311012268066406 time for calcul the mask position with numpy : 0.05013585090637207 nb_pixel_total : 123055 time to create 1 rle with old method : 0.13575339317321777 create new chi : 6.718901872634888 time to delete rle : 0.002775907516479492 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 25901 TO DO : save crop sub photo not yet done ! save time : 1.8300955295562744 map_output_result : {1339803007: (0.0, 'Should be the crop_list due to order', 0), 1339803004: (0.0, 'Should be the crop_list due to order', 0), 1339802998: (0.0, 'Should be the crop_list due to order', 0), 1339802988: (0.0, 'Should be the crop_list due to order', 0), 1339802842: (0.0, 'Should be the crop_list due to order', 0), 1339802837: (0.0, 'Should be the crop_list due to order', 0), 1339802834: (0.0, 'Should be the crop_list due to order', 0), 1339768174: (0.0, 'Should be the crop_list due to order', 0), 1339768171: (0.0, 'Should be the crop_list due to order', 0), 1339768168: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1339803007, 1339803004, 1339802998, 1339802988, 1339802842, 1339802837, 1339802834, 1339768174, 1339768171, 1339768168] Looping around the photos to save general results len do output : 10 /1339803007.Didn't retrieve data . /1339803004.Didn't retrieve data . /1339802998.Didn't retrieve data . /1339802988.Didn't retrieve data . /1339802842.Didn't retrieve data . /1339802837.Didn't retrieve data . /1339802834.Didn't retrieve data . /1339768174.Didn't retrieve data . /1339768171.Didn't retrieve data . /1339768168.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803007', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803004', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802998', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802988', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802842', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802837', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802834', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768174', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768171', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768168', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013861417770385742 save_final save missing photos in datou_result : time spend for datou_step_exec : 160.8751175403595 time spend to save output : 0.014620542526245117 total time spend for step 5 : 160.88973808288574 step6:crop_condition Thu Feb 27 15:51:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 161 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740667882_4139576 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.118133306503296 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740667886_4139576 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.842797040939331 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740667890_4139576 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.503357172012329 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740667894_4139576 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.456573724746704 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740667898_4139576 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.7081389427185059 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740667902_4139576 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.085134506225586 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339803007, 1339803004, 1339802998, 1339802988, 1339802842, 1339802837, 1339802834, 1339768174, 1339768171, 1339768168] Looping around the photos to save general results len do output : 23 /1339935646Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935647Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935648Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935649Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935650Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935651Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935652Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935653Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935654Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935655Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935676Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935680Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935700Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935701Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935702Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935703Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935709Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935717Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339935718Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803007', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803004', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802998', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802988', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802842', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802837', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802834', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768174', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768171', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768168', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 79 time used for this insertion : 0.11487054824829102 save_final save missing photos in datou_result : time spend for datou_step_exec : 26.386719703674316 time spend to save output : 0.1159365177154541 total time spend for step 6 : 26.50265622138977 step7:ventilate_hashtags_in_portfolio Thu Feb 27 15:51:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20881947 get user id for portfolio 20881947 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881947 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','Carton_brun','papier','environnement','kraft','metal','Teint_Dans_La_Masse','flou','autre_refus','plastique','cartonnette','Carton_gris')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881947 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','Carton_brun','papier','environnement','kraft','metal','Teint_Dans_La_Masse','flou','autre_refus','plastique','cartonnette','Carton_gris')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881947 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('mal_croppe','Carton_brun','papier','environnement','kraft','metal','Teint_Dans_La_Masse','flou','autre_refus','plastique','cartonnette','Carton_gris')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20884094,20884095,20884096,20884097,20884098,20884099,20884100,20884101,20884102,20884103,20884104,20884105?tags=mal_croppe,Carton_brun,papier,environnement,kraft,metal,Teint_Dans_La_Masse,flou,autre_refus,plastique,cartonnette,Carton_gris Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1339803007, 1339803004, 1339802998, 1339802988, 1339802842, 1339802837, 1339802834, 1339768174, 1339768171, 1339768168] Looping around the photos to save general results len do output : 1 /20881947. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803007', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803004', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802998', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802988', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802842', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802837', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802834', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768174', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768171', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768168', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014366865158081055 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.9751479625701904 time spend to save output : 0.01473855972290039 total time spend for step 7 : 1.9898865222930908 step8:final Thu Feb 27 15:51:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1339803007: ('0.07376039365091426',), 1339803004: ('0.07376039365091426',), 1339802998: ('0.07376039365091426',), 1339802988: ('0.07376039365091426',), 1339802842: ('0.07376039365091426',), 1339802837: ('0.07376039365091426',), 1339802834: ('0.07376039365091426',), 1339768174: ('0.07376039365091426',), 1339768171: ('0.07376039365091426',), 1339768168: ('0.07376039365091426',)} new output for save of step final : {1339803007: ('0.07376039365091426',), 1339803004: ('0.07376039365091426',), 1339802998: ('0.07376039365091426',), 1339802988: ('0.07376039365091426',), 1339802842: ('0.07376039365091426',), 1339802837: ('0.07376039365091426',), 1339802834: ('0.07376039365091426',), 1339768174: ('0.07376039365091426',), 1339768171: ('0.07376039365091426',), 1339768168: ('0.07376039365091426',)} [1339803007, 1339803004, 1339802998, 1339802988, 1339802842, 1339802837, 1339802834, 1339768174, 1339768171, 1339768168] Looping around the photos to save general results len do output : 10 /1339803007.Didn't retrieve data . /1339803004.Didn't retrieve data . /1339802998.Didn't retrieve data . /1339802988.Didn't retrieve data . /1339802842.Didn't retrieve data . /1339802837.Didn't retrieve data . /1339802834.Didn't retrieve data . /1339768174.Didn't retrieve data . /1339768171.Didn't retrieve data . /1339768168.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803007', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803004', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802998', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802988', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802842', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802837', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802834', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768174', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768171', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768168', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.013924837112426758 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.16648173332214355 time spend to save output : 0.014478206634521484 total time spend for step 8 : 0.18095993995666504 step9:velours_tree Thu Feb 27 15:51:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.13759851455688477 time spend to save output : 7.033348083496094e-05 total time spend for step 9 : 0.13766884803771973 step10:send_mail_cod Thu Feb 27 15:51:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20881947_27-02-2025_15_51_46.pdf 20884094 imagette208840941740667906 20884098 imagette208840981740667906 20884099 imagette208840991740667906 20884101 imagette208841011740667906 20884102 change filename to text .change filename to text .imagette208841021740667906 20884103 change filename to text .change filename to text .imagette208841031740667906 20884104 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208841041740667906 20884095 change filename to text .change filename to text .imagette208840951740667906 20884100 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette208841001740667907 20884105 change filename to text .change filename to text .change filename to text .change filename to text .imagette208841051740667907 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20881947 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20884094,20884095,20884096,20884097,20884098,20884099,20884100,20884101,20884102,20884103,20884104,20884105?tags=mal_croppe,Carton_brun,papier,environnement,kraft,metal,Teint_Dans_La_Masse,flou,autre_refus,plastique,cartonnette,Carton_gris your option no_mail is active, we will not send the real mail to your client args[1339803007] : ('0.07376039365091426',) no score found for photo 1339803007 We are sending mail with results at report@fotonower.com args[1339803004] : ('0.07376039365091426',) no score found for photo 1339803004 We are sending mail with results at report@fotonower.com args[1339802998] : ('0.07376039365091426',) no score found for photo 1339802998 We are sending mail with results at report@fotonower.com args[1339802988] : ('0.07376039365091426',) no score found for photo 1339802988 We are sending mail with results at report@fotonower.com args[1339802842] : ('0.07376039365091426',) no score found for photo 1339802842 We are sending mail with results at report@fotonower.com args[1339802837] : ('0.07376039365091426',) no score found for photo 1339802837 We are sending mail with results at report@fotonower.com args[1339802834] : ('0.07376039365091426',) no score found for photo 1339802834 We are sending mail with results at report@fotonower.com args[1339768174] : ('0.07376039365091426',) no score found for photo 1339768174 We are sending mail with results at report@fotonower.com args[1339768171] : ('0.07376039365091426',) no score found for photo 1339768171 We are sending mail with results at report@fotonower.com args[1339768168] : ('0.07376039365091426',) no score found for photo 1339768168 We are sending mail with results at report@fotonower.com refus_total : 0.07376039365091426 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20881947 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1339803007,1339767578,1339767582,1339803004,1339802842,1339802837,1339802834,1339768174,1339767994,1339768127,1339768171,1339767571,1339768140,1339768163,1339768138,1339802998,1339767590,1339768168,1339768132,1339767992) Found this number of photos: 20 begin to download photo : 1339803007 begin to download photo : 1339802837 begin to download photo : 1339768171 begin to download photo : 1339802998 download finish for photo 1339803007 begin to download photo : 1339767578 download finish for photo 1339802998 begin to download photo : 1339767590 download finish for photo 1339802837 begin to download photo : 1339802834 download finish for photo 1339768171 begin to download photo : 1339767571 download finish for photo 1339767578 begin to download photo : 1339767582 download finish for photo 1339802834 begin to download photo : 1339768174 download finish for photo 1339767571 begin to download photo : 1339768140 download finish for photo 1339767590 begin to download photo : 1339768168 download finish for photo 1339767582 begin to download photo : 1339803004 download finish for photo 1339768140 begin to download photo : 1339768163 download finish for photo 1339768168 begin to download photo : 1339768132 download finish for photo 1339768174 begin to download photo : 1339767994 download finish for photo 1339803004 begin to download photo : 1339802842 download finish for photo 1339768132 begin to download photo : 1339767992 download finish for photo 1339802842 download finish for photo 1339767994 begin to download photo : 1339768127 download finish for photo 1339768163 begin to download photo : 1339768138 download finish for photo 1339767992 download finish for photo 1339768127 download finish for photo 1339768138 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881947_27-02-2025_15_51_46.pdf results_Auto_P20881947_27-02-2025_15_51_46.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881947_27-02-2025_15_51_46.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20881947','results_Auto_P20881947_27-02-2025_15_51_46.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881947_27-02-2025_15_51_46.pdf','pdf','','0.72','0.07376039365091426') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1339803007, 1339803004, 1339802998, 1339802988, 1339802842, 1339802837, 1339802834, 1339768174, 1339768171, 1339768168] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803007', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803004', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802998', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802988', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802842', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802837', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802834', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768174', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768171', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768168', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.01240086555480957 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.1041483879089355 time spend to save output : 0.012772083282470703 total time spend for step 10 : 6.116920471191406 step11:split_time_score Thu Feb 27 15:51:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('18', 26),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 26022025 20881947 Nombre de photos uploadées : 26 / 23040 (0%) 26022025 20881947 Nombre de photos taguées (types de déchets): 0 / 26 (0%) 26022025 20881947 Nombre de photos taguées (volume) : 0 / 26 (0%) elapsed_time : load_data_split_time_score 8.106231689453125e-06 elapsed_time : order_list_meta_photo_and_scores 3.218650817871094e-05 ?????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0016880035400390625 elapsed_time : insert_dashboard_record_day_entry 0.02391958236694336 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881903 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881904 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881905 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881906 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881907 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881908 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881909 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881910 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881911 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881912 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881913 order by id desc limit 1 Qualite : 0.063119307517658 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881914_27-02-2025_15_01_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881914 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881914 AND mptpi.`type`=3726 To do Qualite : 0.15187597077362103 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881915_27-02-2025_15_26_35.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881915 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881915 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881916 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881917 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881918 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881919 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881920 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881921 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881922 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881923 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881924 order by id desc limit 1 Qualite : 0.17112767792302108 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881925_27-02-2025_15_18_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881925 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881925 AND mptpi.`type`=3594 To do Qualite : 0.08562360163737219 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881926_27-02-2025_14_57_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881926 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881926 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881927 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881928 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881929 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881930 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881931 order by id desc limit 1 Qualite : 0.3424234391415626 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881932_27-02-2025_15_47_33.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881932 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881932 AND mptpi.`type`=4200 To do Qualite : 0.08025362176513684 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881933_27-02-2025_15_44_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881933 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881933 AND mptpi.`type`=4211 To do Qualite : 0.2121681261984472 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881934_27-02-2025_15_22_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881934 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881934 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881935 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881936 order by id desc limit 1 Qualite : 0.2051292416450198 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881937_27-02-2025_15_03_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881937 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881937 AND mptpi.`type`=3594 To do Qualite : 0.22323044144596488 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881938_27-02-2025_15_18_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881938 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881938 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881939 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881940 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881941 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881942 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881943 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881944 order by id desc limit 1 Qualite : 0.04719140436512212 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881945_27-02-2025_15_24_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881945 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881945 AND mptpi.`type`=4207 To do Qualite : 0.06795947229191704 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881946_27-02-2025_15_10_24.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881946 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881946 AND mptpi.`type`=4207 To do Qualite : 0.07376039365091426 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881947_27-02-2025_15_51_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881947 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881947 AND mptpi.`type`=3726 To do Qualite : 0.15940719748547566 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881948_27-02-2025_14_47_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881948 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881948 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881949 order by id desc limit 1 Qualite : 0.5400794373803116 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881950_27-02-2025_14_35_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881950 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881950 AND mptpi.`type`=4200 To do Qualite : 0.18727468064128952 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881951 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881951 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881952 order by id desc limit 1 Qualite : 0.05315619982736321 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881953_27-02-2025_14_56_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881953 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881953 AND mptpi.`type`=4207 To do Qualite : 0.09784591310224797 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20881954_27-02-2025_15_37_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20881954 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20881954 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'26022025': {'nb_upload': 26, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1339803007, 1339803004, 1339802998, 1339802988, 1339802842, 1339802837, 1339802834, 1339768174, 1339768171, 1339768168] Looping around the photos to save general results len do output : 1 /20881947Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803007', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339803004', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802998', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802988', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802842', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802837', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339802834', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768174', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768171', None, None, None, None, None, '2619767') ('3459', None, None, None, None, None, None, None, '2619767') ('3459', '20881947', '1339768168', None, None, None, None, None, '2619767') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 1.6123437881469727 save_final save missing photos in datou_result : time spend for datou_step_exec : 30.808367013931274 time spend to save output : 1.6163885593414307 total time spend for step 11 : 32.424755573272705 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 154.87user 159.18system 6:00.83elapsed 87%CPU (0avgtext+0avgdata 4511068maxresident)k 1709856inputs+125952outputs (11554major+14128477minor)pagefaults 0swaps