python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2078403 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 37>10 please execute split_portfolio.py -i 20321739 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2561530'] with mtr_portfolio_ids : ['20321739'] and first list_photo_ids : [] new path : /proc/2078403/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 2.328022003173828 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Fri Feb 7 06:30:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 4023 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-07 06:30:31.750076: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-07 06:30:31.775437: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-07 06:30:31.777047: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f46c4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-07 06:30:31.777098: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-07 06:30:31.779621: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-07 06:30:32.018049: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xe7675d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-07 06:30:32.018090: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-07 06:30:32.018780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 06:30:32.019086: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 06:30:32.021037: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 06:30:32.023085: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 06:30:32.023388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 06:30:32.025557: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 06:30:32.026554: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 06:30:32.030853: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 06:30:32.031932: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 06:30:32.032001: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 06:30:32.032557: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 06:30:32.032572: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 06:30:32.032580: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 06:30:32.033494: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3571 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-07 06:30:32.281725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 06:30:32.281850: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 06:30:32.281867: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 06:30:32.281882: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 06:30:32.281896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 06:30:32.281909: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 06:30:32.281922: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 06:30:32.281936: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 06:30:32.282753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 06:30:32.283711: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-07 06:30:32.283743: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-07 06:30:32.283758: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 06:30:32.283771: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-07 06:30:32.283785: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-07 06:30:32.283799: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-07 06:30:32.283812: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-07 06:30:32.283825: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 06:30:32.284670: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-07 06:30:32.284703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-07 06:30:32.284711: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-07 06:30:32.284717: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-07 06:30:32.285566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3571 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-07 06:30:42.247729: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-07 06:30:42.436118: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-07 06:30:43.940529: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.941116: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.69G (2888603904 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.941825: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.42G (2599743488 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.942411: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.18G (2339769088 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.942441: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:43.943053: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.943071: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:43.959049: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.959071: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:43.959653: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.959667: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:43.968120: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.968146: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:43.968732: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:43.968746: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:44.005861: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.005902: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:44.006486: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.006501: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:44.013216: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.013235: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:44.013817: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.013832: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-07 06:30:44.053868: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.054463: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.056959: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.057598: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.112029: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.112674: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.115287: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.115927: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.151909: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.152554: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.154309: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.154963: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.162746: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.163421: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.165186: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.165818: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.172926: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.173563: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.175194: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.175832: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.206583: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.207254: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.207886: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.208516: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.213037: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.213707: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.231842: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.232495: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.233128: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.233759: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.247380: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.248021: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.248649: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.249280: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.254562: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.255234: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.262342: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.262995: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.276403: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.277036: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.282192: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.282824: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.283470: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.284099: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.284897: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.285530: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.297484: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.298117: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.298756: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.299391: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.300018: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.300643: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.301267: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.301891: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.320860: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.321524: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.355509: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.355592: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-07 06:30:44.356274: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.356915: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.365327: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.366395: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.367523: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.368578: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.378662: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.379706: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.397747: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.398605: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.399456: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.400250: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.404658: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.405325: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.405956: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.406584: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.407852: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.419272: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.419950: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.430507: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.431223: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.431872: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.432504: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.433161: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-07 06:30:44.433791: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.99G (3209560064 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 22 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 26 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 13 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 5 Detection mask done ! Trying to reset tf kernel 2079107 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 886 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3272 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.2453775405883789 nb_pixel_total : 3251537 time to create 1 rle with new method : 0.44069409370422363 length of segment : 2823 time for calcul the mask position with numpy : 0.0003159046173095703 nb_pixel_total : 9399 time to create 1 rle with old method : 0.012521982192993164 length of segment : 189 time for calcul the mask position with numpy : 0.002226114273071289 nb_pixel_total : 81981 time to create 1 rle with old method : 0.10050368309020996 length of segment : 400 time for calcul the mask position with numpy : 0.002574443817138672 nb_pixel_total : 120769 time to create 1 rle with old method : 0.15615153312683105 length of segment : 599 time for calcul the mask position with numpy : 0.0009942054748535156 nb_pixel_total : 36042 time to create 1 rle with old method : 0.041153669357299805 length of segment : 253 time for calcul the mask position with numpy : 0.0005800724029541016 nb_pixel_total : 6998 time to create 1 rle with old method : 0.011385440826416016 length of segment : 93 time for calcul the mask position with numpy : 0.0005996227264404297 nb_pixel_total : 10393 time to create 1 rle with old method : 0.012532472610473633 length of segment : 84 time for calcul the mask position with numpy : 0.0041997432708740234 nb_pixel_total : 118566 time to create 1 rle with old method : 0.13232827186584473 length of segment : 464 time for calcul the mask position with numpy : 0.00033736228942871094 nb_pixel_total : 3193 time to create 1 rle with old method : 0.005170345306396484 length of segment : 137 time for calcul the mask position with numpy : 0.0017681121826171875 nb_pixel_total : 42694 time to create 1 rle with old method : 0.049123287200927734 length of segment : 226 time for calcul the mask position with numpy : 0.004372358322143555 nb_pixel_total : 123923 time to create 1 rle with old method : 0.13410401344299316 length of segment : 526 time for calcul the mask position with numpy : 0.0014176368713378906 nb_pixel_total : 30701 time to create 1 rle with old method : 0.03566122055053711 length of segment : 313 time for calcul the mask position with numpy : 0.002463817596435547 nb_pixel_total : 51107 time to create 1 rle with old method : 0.0573117733001709 length of segment : 367 time for calcul the mask position with numpy : 0.005469083786010742 nb_pixel_total : 73759 time to create 1 rle with old method : 0.08326005935668945 length of segment : 724 time for calcul the mask position with numpy : 0.001573801040649414 nb_pixel_total : 45417 time to create 1 rle with old method : 0.05071377754211426 length of segment : 326 time for calcul the mask position with numpy : 0.0010268688201904297 nb_pixel_total : 29224 time to create 1 rle with old method : 0.039157867431640625 length of segment : 293 time for calcul the mask position with numpy : 0.00033402442932128906 nb_pixel_total : 3201 time to create 1 rle with old method : 0.0043790340423583984 length of segment : 65 time for calcul the mask position with numpy : 0.002534151077270508 nb_pixel_total : 22558 time to create 1 rle with old method : 0.027286529541015625 length of segment : 298 time for calcul the mask position with numpy : 0.0028672218322753906 nb_pixel_total : 109037 time to create 1 rle with old method : 0.11901688575744629 length of segment : 493 time for calcul the mask position with numpy : 0.006786823272705078 nb_pixel_total : 182843 time to create 1 rle with new method : 0.013079166412353516 length of segment : 640 time for calcul the mask position with numpy : 0.0017170906066894531 nb_pixel_total : 51974 time to create 1 rle with old method : 0.05849146842956543 length of segment : 381 time for calcul the mask position with numpy : 0.0040569305419921875 nb_pixel_total : 122440 time to create 1 rle with old method : 0.13043785095214844 length of segment : 465 time for calcul the mask position with numpy : 0.000982046127319336 nb_pixel_total : 49006 time to create 1 rle with old method : 0.05372452735900879 length of segment : 364 time for calcul the mask position with numpy : 0.002079010009765625 nb_pixel_total : 116481 time to create 1 rle with old method : 0.1251521110534668 length of segment : 466 time for calcul the mask position with numpy : 0.0002410411834716797 nb_pixel_total : 3784 time to create 1 rle with old method : 0.004874229431152344 length of segment : 63 time for calcul the mask position with numpy : 0.3560903072357178 nb_pixel_total : 4614413 time to create 1 rle with new method : 0.6418492794036865 length of segment : 3799 time for calcul the mask position with numpy : 0.00029206275939941406 nb_pixel_total : 1266 time to create 1 rle with old method : 0.002024412155151367 length of segment : 36 time for calcul the mask position with numpy : 0.0005142688751220703 nb_pixel_total : 12417 time to create 1 rle with old method : 0.01850581169128418 length of segment : 102 time for calcul the mask position with numpy : 0.004411458969116211 nb_pixel_total : 131146 time to create 1 rle with old method : 0.17057418823242188 length of segment : 434 time for calcul the mask position with numpy : 0.0030384063720703125 nb_pixel_total : 74095 time to create 1 rle with old method : 0.08530116081237793 length of segment : 317 time for calcul the mask position with numpy : 0.0005731582641601562 nb_pixel_total : 9009 time to create 1 rle with old method : 0.017864465713500977 length of segment : 89 time for calcul the mask position with numpy : 0.0042285919189453125 nb_pixel_total : 128466 time to create 1 rle with old method : 0.13790607452392578 length of segment : 320 time for calcul the mask position with numpy : 0.0009398460388183594 nb_pixel_total : 21672 time to create 1 rle with old method : 0.024502992630004883 length of segment : 227 time for calcul the mask position with numpy : 0.002453327178955078 nb_pixel_total : 24850 time to create 1 rle with old method : 0.027622699737548828 length of segment : 275 time for calcul the mask position with numpy : 0.007899761199951172 nb_pixel_total : 323677 time to create 1 rle with new method : 0.014265060424804688 length of segment : 406 time for calcul the mask position with numpy : 0.0417172908782959 nb_pixel_total : 839704 time to create 1 rle with new method : 0.7817397117614746 length of segment : 1990 time for calcul the mask position with numpy : 0.0010309219360351562 nb_pixel_total : 23612 time to create 1 rle with old method : 0.027753353118896484 length of segment : 255 time for calcul the mask position with numpy : 0.0005850791931152344 nb_pixel_total : 6824 time to create 1 rle with old method : 0.008064985275268555 length of segment : 98 time for calcul the mask position with numpy : 0.0004570484161376953 nb_pixel_total : 10006 time to create 1 rle with old method : 0.01218724250793457 length of segment : 129 time for calcul the mask position with numpy : 0.00064849853515625 nb_pixel_total : 12763 time to create 1 rle with old method : 0.015049457550048828 length of segment : 129 time for calcul the mask position with numpy : 0.002683877944946289 nb_pixel_total : 136866 time to create 1 rle with old method : 0.14220499992370605 length of segment : 323 time for calcul the mask position with numpy : 0.0011286735534667969 nb_pixel_total : 39573 time to create 1 rle with old method : 0.04311990737915039 length of segment : 306 time for calcul the mask position with numpy : 0.003774881362915039 nb_pixel_total : 56230 time to create 1 rle with old method : 0.06252288818359375 length of segment : 330 time for calcul the mask position with numpy : 0.01927471160888672 nb_pixel_total : 712298 time to create 1 rle with new method : 0.07709789276123047 length of segment : 1365 time for calcul the mask position with numpy : 0.00016164779663085938 nb_pixel_total : 3062 time to create 1 rle with old method : 0.003648519515991211 length of segment : 55 time for calcul the mask position with numpy : 0.014980077743530273 nb_pixel_total : 746440 time to create 1 rle with new method : 0.348036527633667 length of segment : 1361 time for calcul the mask position with numpy : 0.0013458728790283203 nb_pixel_total : 31329 time to create 1 rle with old method : 0.03555607795715332 length of segment : 323 time for calcul the mask position with numpy : 0.000804901123046875 nb_pixel_total : 22427 time to create 1 rle with old method : 0.026077985763549805 length of segment : 105 time for calcul the mask position with numpy : 0.0002014636993408203 nb_pixel_total : 2634 time to create 1 rle with old method : 0.0031180381774902344 length of segment : 68 time for calcul the mask position with numpy : 0.0005428791046142578 nb_pixel_total : 15370 time to create 1 rle with old method : 0.018555402755737305 length of segment : 126 time for calcul the mask position with numpy : 0.001373291015625 nb_pixel_total : 42863 time to create 1 rle with old method : 0.04866957664489746 length of segment : 286 time for calcul the mask position with numpy : 0.004759550094604492 nb_pixel_total : 154215 time to create 1 rle with new method : 0.00831294059753418 length of segment : 556 time for calcul the mask position with numpy : 0.00483250617980957 nb_pixel_total : 187262 time to create 1 rle with new method : 0.008220195770263672 length of segment : 707 time for calcul the mask position with numpy : 0.0002391338348388672 nb_pixel_total : 3768 time to create 1 rle with old method : 0.004450321197509766 length of segment : 67 time for calcul the mask position with numpy : 0.003910064697265625 nb_pixel_total : 146890 time to create 1 rle with old method : 0.1643838882446289 length of segment : 439 time for calcul the mask position with numpy : 0.0007147789001464844 nb_pixel_total : 25831 time to create 1 rle with old method : 0.038320302963256836 length of segment : 148 time for calcul the mask position with numpy : 0.001642465591430664 nb_pixel_total : 36497 time to create 1 rle with old method : 0.04796862602233887 length of segment : 220 time for calcul the mask position with numpy : 0.0009133815765380859 nb_pixel_total : 30875 time to create 1 rle with old method : 0.03418469429016113 length of segment : 348 time for calcul the mask position with numpy : 0.0012810230255126953 nb_pixel_total : 33229 time to create 1 rle with old method : 0.037523746490478516 length of segment : 357 time for calcul the mask position with numpy : 0.0022268295288085938 nb_pixel_total : 66216 time to create 1 rle with old method : 0.07585668563842773 length of segment : 363 time for calcul the mask position with numpy : 0.00014090538024902344 nb_pixel_total : 859 time to create 1 rle with old method : 0.0010941028594970703 length of segment : 40 time for calcul the mask position with numpy : 0.000507354736328125 nb_pixel_total : 30340 time to create 1 rle with old method : 0.03564286231994629 length of segment : 183 time for calcul the mask position with numpy : 0.0009081363677978516 nb_pixel_total : 43290 time to create 1 rle with old method : 0.0471806526184082 length of segment : 342 time for calcul the mask position with numpy : 0.0001685619354248047 nb_pixel_total : 1178 time to create 1 rle with old method : 0.001508474349975586 length of segment : 52 time for calcul the mask position with numpy : 0.0005717277526855469 nb_pixel_total : 9368 time to create 1 rle with old method : 0.015414953231811523 length of segment : 91 time for calcul the mask position with numpy : 0.0003681182861328125 nb_pixel_total : 4697 time to create 1 rle with old method : 0.007767438888549805 length of segment : 87 time for calcul the mask position with numpy : 0.002863645553588867 nb_pixel_total : 62803 time to create 1 rle with old method : 0.07340860366821289 length of segment : 339 time for calcul the mask position with numpy : 0.0020766258239746094 nb_pixel_total : 62541 time to create 1 rle with old method : 0.07070684432983398 length of segment : 419 time for calcul the mask position with numpy : 0.00024700164794921875 nb_pixel_total : 3627 time to create 1 rle with old method : 0.0043792724609375 length of segment : 56 time for calcul the mask position with numpy : 0.007343292236328125 nb_pixel_total : 236192 time to create 1 rle with new method : 0.015438079833984375 length of segment : 648 time for calcul the mask position with numpy : 0.0003116130828857422 nb_pixel_total : 5780 time to create 1 rle with old method : 0.006333351135253906 length of segment : 95 time for calcul the mask position with numpy : 0.0009658336639404297 nb_pixel_total : 9641 time to create 1 rle with old method : 0.011531352996826172 length of segment : 247 time for calcul the mask position with numpy : 0.00016021728515625 nb_pixel_total : 1529 time to create 1 rle with old method : 0.0017976760864257812 length of segment : 62 time for calcul the mask position with numpy : 0.00032329559326171875 nb_pixel_total : 5633 time to create 1 rle with old method : 0.006676673889160156 length of segment : 126 time for calcul the mask position with numpy : 0.002513408660888672 nb_pixel_total : 60829 time to create 1 rle with old method : 0.06968832015991211 length of segment : 366 time for calcul the mask position with numpy : 0.00048470497131347656 nb_pixel_total : 11801 time to create 1 rle with old method : 0.013299703598022461 length of segment : 105 time for calcul the mask position with numpy : 0.0003275871276855469 nb_pixel_total : 12657 time to create 1 rle with old method : 0.01462101936340332 length of segment : 125 time for calcul the mask position with numpy : 0.013944864273071289 nb_pixel_total : 513597 time to create 1 rle with new method : 0.037212371826171875 length of segment : 556 time for calcul the mask position with numpy : 0.0006513595581054688 nb_pixel_total : 12369 time to create 1 rle with old method : 0.014171123504638672 length of segment : 107 time for calcul the mask position with numpy : 0.005750179290771484 nb_pixel_total : 212344 time to create 1 rle with new method : 0.010827302932739258 length of segment : 436 time for calcul the mask position with numpy : 0.0010247230529785156 nb_pixel_total : 62317 time to create 1 rle with old method : 0.06493330001831055 length of segment : 335 time for calcul the mask position with numpy : 0.00019240379333496094 nb_pixel_total : 1360 time to create 1 rle with old method : 0.0015978813171386719 length of segment : 63 time for calcul the mask position with numpy : 0.002585887908935547 nb_pixel_total : 61896 time to create 1 rle with old method : 0.06980347633361816 length of segment : 356 time for calcul the mask position with numpy : 0.006472587585449219 nb_pixel_total : 268886 time to create 1 rle with new method : 0.008493423461914062 length of segment : 851 time for calcul the mask position with numpy : 0.00019407272338867188 nb_pixel_total : 2196 time to create 1 rle with old method : 0.0027570724487304688 length of segment : 55 time for calcul the mask position with numpy : 0.0019998550415039062 nb_pixel_total : 34575 time to create 1 rle with old method : 0.03833150863647461 length of segment : 470 time for calcul the mask position with numpy : 0.0011775493621826172 nb_pixel_total : 37535 time to create 1 rle with old method : 0.045104026794433594 length of segment : 295 time for calcul the mask position with numpy : 0.0014295578002929688 nb_pixel_total : 52839 time to create 1 rle with old method : 0.05746197700500488 length of segment : 362 time for calcul the mask position with numpy : 0.00016117095947265625 nb_pixel_total : 3842 time to create 1 rle with old method : 0.005077838897705078 length of segment : 49 time for calcul the mask position with numpy : 0.0037631988525390625 nb_pixel_total : 104384 time to create 1 rle with old method : 0.11278414726257324 length of segment : 458 time for calcul the mask position with numpy : 0.0008642673492431641 nb_pixel_total : 12016 time to create 1 rle with old method : 0.014421463012695312 length of segment : 155 time for calcul the mask position with numpy : 0.002085447311401367 nb_pixel_total : 64724 time to create 1 rle with old method : 0.07480001449584961 length of segment : 479 time for calcul the mask position with numpy : 0.0002663135528564453 nb_pixel_total : 4067 time to create 1 rle with old method : 0.005039691925048828 length of segment : 93 time for calcul the mask position with numpy : 0.010818004608154297 nb_pixel_total : 308302 time to create 1 rle with new method : 0.015949726104736328 length of segment : 891 time for calcul the mask position with numpy : 0.009148836135864258 nb_pixel_total : 377285 time to create 1 rle with new method : 0.012892723083496094 length of segment : 624 time for calcul the mask position with numpy : 0.004087209701538086 nb_pixel_total : 152505 time to create 1 rle with new method : 0.005932331085205078 length of segment : 572 time for calcul the mask position with numpy : 0.00011730194091796875 nb_pixel_total : 3118 time to create 1 rle with old method : 0.003450155258178711 length of segment : 50 time for calcul the mask position with numpy : 0.00020647048950195312 nb_pixel_total : 1575 time to create 1 rle with old method : 0.0019526481628417969 length of segment : 81 time for calcul the mask position with numpy : 0.0009047985076904297 nb_pixel_total : 21094 time to create 1 rle with old method : 0.02264118194580078 length of segment : 263 time for calcul the mask position with numpy : 0.0007901191711425781 nb_pixel_total : 58929 time to create 1 rle with old method : 0.06426644325256348 length of segment : 390 time for calcul the mask position with numpy : 0.002072572708129883 nb_pixel_total : 152922 time to create 1 rle with new method : 0.005299568176269531 length of segment : 577 time for calcul the mask position with numpy : 0.0019845962524414062 nb_pixel_total : 53874 time to create 1 rle with old method : 0.05924534797668457 length of segment : 580 time for calcul the mask position with numpy : 0.0012574195861816406 nb_pixel_total : 49395 time to create 1 rle with old method : 0.05707693099975586 length of segment : 341 time for calcul the mask position with numpy : 0.0008661746978759766 nb_pixel_total : 49472 time to create 1 rle with old method : 0.056877851486206055 length of segment : 347 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 3788 time to create 1 rle with old method : 0.004665374755859375 length of segment : 54 time for calcul the mask position with numpy : 0.00032138824462890625 nb_pixel_total : 6222 time to create 1 rle with old method : 0.007394552230834961 length of segment : 117 time for calcul the mask position with numpy : 0.010970830917358398 nb_pixel_total : 251965 time to create 1 rle with new method : 0.09449410438537598 length of segment : 732 time for calcul the mask position with numpy : 0.00552821159362793 nb_pixel_total : 251830 time to create 1 rle with new method : 0.009166479110717773 length of segment : 776 time for calcul the mask position with numpy : 0.0010535717010498047 nb_pixel_total : 41241 time to create 1 rle with old method : 0.045876502990722656 length of segment : 441 time for calcul the mask position with numpy : 0.0037398338317871094 nb_pixel_total : 181456 time to create 1 rle with new method : 0.005927562713623047 length of segment : 471 time for calcul the mask position with numpy : 0.005284786224365234 nb_pixel_total : 335487 time to create 1 rle with new method : 0.007294178009033203 length of segment : 482 time for calcul the mask position with numpy : 0.0010957717895507812 nb_pixel_total : 65105 time to create 1 rle with old method : 0.07473874092102051 length of segment : 288 time for calcul the mask position with numpy : 0.005276679992675781 nb_pixel_total : 425743 time to create 1 rle with new method : 0.010007858276367188 length of segment : 523 time for calcul the mask position with numpy : 0.00018548965454101562 nb_pixel_total : 8554 time to create 1 rle with old method : 0.010199308395385742 length of segment : 108 time for calcul the mask position with numpy : 0.0001342296600341797 nb_pixel_total : 931 time to create 1 rle with old method : 0.0011339187622070312 length of segment : 51 time for calcul the mask position with numpy : 0.0010666847229003906 nb_pixel_total : 35664 time to create 1 rle with old method : 0.04159355163574219 length of segment : 283 time for calcul the mask position with numpy : 0.000942230224609375 nb_pixel_total : 33987 time to create 1 rle with old method : 0.039217233657836914 length of segment : 306 time for calcul the mask position with numpy : 0.0002338886260986328 nb_pixel_total : 2441 time to create 1 rle with old method : 0.002905607223510742 length of segment : 96 time for calcul the mask position with numpy : 0.0001544952392578125 nb_pixel_total : 1226 time to create 1 rle with old method : 0.0014808177947998047 length of segment : 65 time for calcul the mask position with numpy : 0.01026153564453125 nb_pixel_total : 427515 time to create 1 rle with new method : 0.015067338943481445 length of segment : 635 time for calcul the mask position with numpy : 0.0007715225219726562 nb_pixel_total : 17486 time to create 1 rle with old method : 0.020696401596069336 length of segment : 191 time for calcul the mask position with numpy : 0.000659942626953125 nb_pixel_total : 12788 time to create 1 rle with old method : 0.015842199325561523 length of segment : 206 time for calcul the mask position with numpy : 0.007067203521728516 nb_pixel_total : 170478 time to create 1 rle with new method : 0.011953115463256836 length of segment : 939 time for calcul the mask position with numpy : 0.0020551681518554688 nb_pixel_total : 42525 time to create 1 rle with old method : 0.052977800369262695 length of segment : 399 time for calcul the mask position with numpy : 0.00035119056701660156 nb_pixel_total : 18868 time to create 1 rle with old method : 0.022538185119628906 length of segment : 183 time for calcul the mask position with numpy : 0.00871729850769043 nb_pixel_total : 368648 time to create 1 rle with new method : 0.01537775993347168 length of segment : 613 time for calcul the mask position with numpy : 0.00024127960205078125 nb_pixel_total : 7512 time to create 1 rle with old method : 0.009091854095458984 length of segment : 99 time for calcul the mask position with numpy : 0.0007328987121582031 nb_pixel_total : 12223 time to create 1 rle with old method : 0.01442718505859375 length of segment : 246 time for calcul the mask position with numpy : 0.0004787445068359375 nb_pixel_total : 22068 time to create 1 rle with old method : 0.024822235107421875 length of segment : 166 time for calcul the mask position with numpy : 0.0003535747528076172 nb_pixel_total : 10029 time to create 1 rle with old method : 0.012604475021362305 length of segment : 73 time for calcul the mask position with numpy : 0.0012607574462890625 nb_pixel_total : 30288 time to create 1 rle with old method : 0.034729719161987305 length of segment : 449 time for calcul the mask position with numpy : 0.0015985965728759766 nb_pixel_total : 63675 time to create 1 rle with old method : 0.07325077056884766 length of segment : 354 time for calcul the mask position with numpy : 0.0022132396697998047 nb_pixel_total : 92045 time to create 1 rle with old method : 0.10457110404968262 length of segment : 421 time for calcul the mask position with numpy : 0.0015642642974853516 nb_pixel_total : 87817 time to create 1 rle with old method : 0.10063433647155762 length of segment : 465 time for calcul the mask position with numpy : 0.6811923980712891 nb_pixel_total : 4757713 time to create 1 rle with new method : 0.8896594047546387 length of segment : 2702 time for calcul the mask position with numpy : 0.0018317699432373047 nb_pixel_total : 127417 time to create 1 rle with old method : 0.13553476333618164 length of segment : 554 time for calcul the mask position with numpy : 0.0004401206970214844 nb_pixel_total : 25793 time to create 1 rle with old method : 0.029709815979003906 length of segment : 207 time for calcul the mask position with numpy : 0.0006570816040039062 nb_pixel_total : 38916 time to create 1 rle with old method : 0.04145455360412598 length of segment : 174 time for calcul the mask position with numpy : 0.0009069442749023438 nb_pixel_total : 22604 time to create 1 rle with old method : 0.025392770767211914 length of segment : 187 time spent for convertir_results : 22.448713302612305 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 139 chid ids of type : 3663 Number RLEs to save : 54786 save missing photos in datou_result : time spend for datou_step_exec : 78.60428953170776 time spend to save output : 12.550549983978271 total time spend for step 1 : 91.15483951568604 step2:crop_condition Fri Feb 7 06:32:00 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 139 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 19 About to insert : list_path_to_insert length 19 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 37 About to insert : list_path_to_insert length 37 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335352207, 1335352206, 1335352204, 1335352200, 1335350972, 1335350969, 1335350967, 1335348492, 1335348488, 1335348485] Looping around the photos to save general results len do output : 85 /-3659049379Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049400Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049493Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049394Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049390Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049391Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049388Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049393Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049385Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049450Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049451Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049467Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049465Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049481Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049484Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049504Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049494Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049496Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049508Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049387Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049430Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049452Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049466Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049473Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049505Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049384Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049389Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049395Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049386Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049382Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049383Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049401Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049410Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049403Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049405Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049406Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049411Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049423Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049431Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049424Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049433Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049418Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049438Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049475Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049463Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049470Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049483Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049495Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049503Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049510Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049428Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049440Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049498Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049381Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049404Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049472Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049478Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049487Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049507Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3659049474Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352207', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352206', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352204', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352200', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350972', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350969', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350967', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348492', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348488', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348485', None, None, None, None, None, '2561530') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 265 time used for this insertion : 0.05625414848327637 save_final save missing photos in datou_result : time spend for datou_step_exec : 50.29430031776428 time spend to save output : 0.1170961856842041 total time spend for step 2 : 50.411396503448486 step3:thcl Fri Feb 7 06:32:50 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.015115022659301758 time to convert the images to numpy array : 0.2657797336578369 time to import caffe and check if the image exist : 0.015117883682250977 time to convert the images to numpy array : 0.5068044662475586 time to import caffe and check if the image exist : 0.010134696960449219 time to convert the images to numpy array : 0.5198414325714111 time to import caffe and check if the image exist : 0.012229442596435547 time to convert the images to numpy array : 0.519773006439209 time to import caffe and check if the image exist : 0.023185014724731445 time to convert the images to numpy array : 0.5137341022491455 time to import caffe and check if the image exist : 0.016091585159301758 time to convert the images to numpy array : 0.5240325927734375 time to import caffe and check if the image exist : 0.017479896545410156 time to convert the images to numpy array : 0.706597089767456 time to import caffe and check if the image exist : 0.013597488403320312 time to convert the images to numpy array : 0.7118492126464844 time to import caffe and check if the image exist : 0.01412343978881836 time to convert the images to numpy array : 0.717033863067627 time to import caffe and check if the image exist : 0.017232656478881836 time to convert the images to numpy array : 0.834822416305542 total time to convert the images to numpy array : 1.1873953342437744 list photo_ids error: [] list photo_ids correct : [-3659049428, -3659049429, -3659049440, -3659049442, -3659049498, -3659049381, -3659049404, -3659049472, -3659049458, -3659049455, -3659049454, -3659049470, -3659049483, -3659049495, -3659049503, -3659049509, -3659049510, -3659049419, -3659049478, -3659049487, -3659049507, -3659049474, -3659049379, -3659049400, -3659049441, -3659049469, -3659049459, -3659049493, -3659049375, -3659049374, -3659049394, -3659049414, -3659049430, -3659049446, -3659049452, -3659049466, -3659049473, -3659049505, -3659049384, -3659049389, -3659049390, -3659049391, -3659049388, -3659049393, -3659049385, -3659049422, -3659049450, -3659049451, -3659049467, -3659049426, -3659049449, -3659049438, -3659049439, -3659049448, -3659049461, -3659049475, -3659049457, -3659049463, -3659049403, -3659049405, -3659049406, -3659049411, -3659049423, -3659049431, -3659049424, -3659049433, -3659049418, -3659049465, -3659049481, -3659049484, -3659049504, -3659049494, -3659049496, -3659049508, -3659049387, -3659049413, -3659049395, -3659049386, -3659049382, -3659049376, -3659049383, -3659049401, -3659049415, -3659049412, -3659049410] number of photos to traite : 85 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 1858 wait 20 seconds l 3637 free memory gpu now : 1858 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2832 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.2021799087524414 time used to do the prediction : 0.25022411346435547 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2830 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2830 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.7742841243743896 time used to do the prediction : 0.18988561630249023 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 2.384185791015625e-05 save missing photos in datou_result : time spend for datou_step_exec : 31.816211223602295 time spend to save output : 0.0008389949798583984 total time spend for step 3 : 31.817050218582153 step4:merge_mask_thcl_custom Fri Feb 7 06:33:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 139 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1335352207, 1335352206, 1335352204, 1335352200, 1335350972, 1335350969, 1335350967, 1335348492, 1335348488, 1335348485] Looping around the photos to save general results len do output : 10 /1335352207Didn't retrieve data .Didn't retrieve data . /1335352206Didn't retrieve data .Didn't retrieve data . /1335352204Didn't retrieve data .Didn't retrieve data . /1335352200Didn't retrieve data .Didn't retrieve data . /1335350972Didn't retrieve data .Didn't retrieve data . /1335350969Didn't retrieve data .Didn't retrieve data . /1335350967Didn't retrieve data .Didn't retrieve data . /1335348492Didn't retrieve data .Didn't retrieve data . /1335348488Didn't retrieve data .Didn't retrieve data . /1335348485Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352207', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352206', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352204', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352200', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350972', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350969', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350967', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348492', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348488', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348485', None, None, None, None, None, '2561530') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.016026735305786133 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06445741653442383 time spend to save output : 0.016638517379760742 total time spend for step 4 : 0.08109593391418457 step5:rle_unique_nms_with_priority Fri Feb 7 06:33:22 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 4.460039377212524 time for calcul the mask position with numpy : 0.18880033493041992 nb_pixel_total : 3588380 time to create 1 rle with new method : 0.7719631195068359 time for calcul the mask position with numpy : 0.034566402435302734 nb_pixel_total : 81814 time to create 1 rle with old method : 0.08595824241638184 time for calcul the mask position with numpy : 0.0340421199798584 nb_pixel_total : 9379 time to create 1 rle with old method : 0.010732412338256836 time for calcul the mask position with numpy : 0.4053318500518799 nb_pixel_total : 3249091 time to create 1 rle with new method : 0.5520462989807129 time for calcul the mask position with numpy : 0.028876066207885742 nb_pixel_total : 121576 time to create 1 rle with old method : 0.1311945915222168 create new chi : 2.3078248500823975 time to delete rle : 0.009946823120117188 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 10110 TO DO : save crop sub photo not yet done ! save time : 3.55429744720459 nb_obj : 19 nb_hashtags : 7 time to prepare the origin masks : 14.15296721458435 time for calcul the mask position with numpy : 0.5378141403198242 nb_pixel_total : 6295560 time to create 1 rle with new method : 0.6742250919342041 time for calcul the mask position with numpy : 0.03408646583557129 nb_pixel_total : 182635 time to create 1 rle with new method : 0.5558874607086182 time for calcul the mask position with numpy : 0.039225101470947266 nb_pixel_total : 2015 time to create 1 rle with old method : 0.0028429031372070312 time for calcul the mask position with numpy : 0.038483619689941406 nb_pixel_total : 7639 time to create 1 rle with old method : 0.009181022644042969 time for calcul the mask position with numpy : 0.02435159683227539 nb_pixel_total : 118506 time to create 1 rle with old method : 0.12557148933410645 time for calcul the mask position with numpy : 0.02411675453186035 nb_pixel_total : 45290 time to create 1 rle with old method : 0.054004669189453125 time for calcul the mask position with numpy : 0.03740096092224121 nb_pixel_total : 1909 time to create 1 rle with old method : 0.00229644775390625 time for calcul the mask position with numpy : 0.03596639633178711 nb_pixel_total : 2522 time to create 1 rle with old method : 0.005557537078857422 time for calcul the mask position with numpy : 0.040032148361206055 nb_pixel_total : 36026 time to create 1 rle with old method : 0.041036128997802734 time for calcul the mask position with numpy : 0.03323006629943848 nb_pixel_total : 48952 time to create 1 rle with old method : 0.05518221855163574 time for calcul the mask position with numpy : 0.03340268135070801 nb_pixel_total : 1661 time to create 1 rle with old method : 0.004282474517822266 time for calcul the mask position with numpy : 0.03432798385620117 nb_pixel_total : 42565 time to create 1 rle with old method : 0.04733157157897949 time for calcul the mask position with numpy : 0.035646677017211914 nb_pixel_total : 30517 time to create 1 rle with old method : 0.03599286079406738 time for calcul the mask position with numpy : 0.03300046920776367 nb_pixel_total : 2018 time to create 1 rle with old method : 0.002463102340698242 time for calcul the mask position with numpy : 0.03575325012207031 nb_pixel_total : 116215 time to create 1 rle with old method : 0.12329673767089844 time for calcul the mask position with numpy : 0.022813081741333008 nb_pixel_total : 73571 time to create 1 rle with old method : 0.07919478416442871 time for calcul the mask position with numpy : 0.022931575775146484 nb_pixel_total : 3190 time to create 1 rle with old method : 0.004068851470947266 time for calcul the mask position with numpy : 0.02322840690612793 nb_pixel_total : 10347 time to create 1 rle with old method : 0.011244773864746094 time for calcul the mask position with numpy : 0.022299528121948242 nb_pixel_total : 6914 time to create 1 rle with old method : 0.007587432861328125 time for calcul the mask position with numpy : 0.02248835563659668 nb_pixel_total : 22188 time to create 1 rle with old method : 0.028147220611572266 create new chi : 3.073317050933838 time to delete rle : 0.001735687255859375 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 13165 TO DO : save crop sub photo not yet done ! save time : 1.9278156757354736 nb_obj : 3 nb_hashtags : 1 time to prepare the origin masks : 4.152330160140991 time for calcul the mask position with numpy : 0.04993033409118652 nb_pixel_total : 2439064 time to create 1 rle with new method : 0.554715633392334 time for calcul the mask position with numpy : 0.025096893310546875 nb_pixel_total : 1254 time to create 1 rle with old method : 0.0015430450439453125 time for calcul the mask position with numpy : 0.2588834762573242 nb_pixel_total : 4606181 time to create 1 rle with new method : 2.747525215148926 time for calcul the mask position with numpy : 0.025380611419677734 nb_pixel_total : 3741 time to create 1 rle with old method : 0.004328489303588867 create new chi : 3.732656240463257 time to delete rle : 0.0009260177612304688 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 9992 TO DO : save crop sub photo not yet done ! save time : 4.949974536895752 nb_obj : 18 nb_hashtags : 7 time to prepare the origin masks : 12.246249675750732 time for calcul the mask position with numpy : 0.34147000312805176 nb_pixel_total : 4975225 time to create 1 rle with new method : 0.7526013851165771 time for calcul the mask position with numpy : 0.02191781997680664 nb_pixel_total : 128044 time to create 1 rle with old method : 0.13669586181640625 time for calcul the mask position with numpy : 0.022554397583007812 nb_pixel_total : 9937 time to create 1 rle with old method : 0.010672330856323242 time for calcul the mask position with numpy : 0.021531343460083008 nb_pixel_total : 21652 time to create 1 rle with old method : 0.023061513900756836 time for calcul the mask position with numpy : 0.021683454513549805 nb_pixel_total : 28714 time to create 1 rle with old method : 0.03155207633972168 time for calcul the mask position with numpy : 0.021463394165039062 nb_pixel_total : 2553 time to create 1 rle with old method : 0.0029456615447998047 time for calcul the mask position with numpy : 0.022083759307861328 nb_pixel_total : 134217 time to create 1 rle with old method : 0.1544942855834961 time for calcul the mask position with numpy : 0.023566246032714844 nb_pixel_total : 12697 time to create 1 rle with old method : 0.015254735946655273 time for calcul the mask position with numpy : 0.026007413864135742 nb_pixel_total : 323272 time to create 1 rle with new method : 1.024501085281372 time for calcul the mask position with numpy : 0.020222902297973633 nb_pixel_total : 6752 time to create 1 rle with old method : 0.007865667343139648 time for calcul the mask position with numpy : 0.021528005599975586 nb_pixel_total : 3126 time to create 1 rle with old method : 0.003627300262451172 time for calcul the mask position with numpy : 0.02957320213317871 nb_pixel_total : 347035 time to create 1 rle with new method : 0.7645752429962158 time for calcul the mask position with numpy : 0.0416722297668457 nb_pixel_total : 723394 time to create 1 rle with new method : 0.44913220405578613 time for calcul the mask position with numpy : 0.03626585006713867 nb_pixel_total : 73971 time to create 1 rle with old method : 0.07999539375305176 time for calcul the mask position with numpy : 0.03687691688537598 nb_pixel_total : 130778 time to create 1 rle with old method : 0.14094901084899902 time for calcul the mask position with numpy : 0.04019951820373535 nb_pixel_total : 24588 time to create 1 rle with old method : 0.027327537536621094 time for calcul the mask position with numpy : 0.034944772720336914 nb_pixel_total : 55963 time to create 1 rle with old method : 0.06033468246459961 time for calcul the mask position with numpy : 0.03413057327270508 nb_pixel_total : 8937 time to create 1 rle with old method : 0.01026773452758789 time for calcul the mask position with numpy : 0.03451657295227051 nb_pixel_total : 39385 time to create 1 rle with old method : 0.04230999946594238 create new chi : 4.704296350479126 time to delete rle : 0.0013697147369384766 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 15600 TO DO : save crop sub photo not yet done ! save time : 3.656926393508911 nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 8.50308632850647 time for calcul the mask position with numpy : 0.2156054973602295 nb_pixel_total : 6330763 time to create 1 rle with new method : 0.556786298751831 time for calcul the mask position with numpy : 0.022727489471435547 nb_pixel_total : 158125 time to create 1 rle with new method : 0.46956920623779297 time for calcul the mask position with numpy : 0.022238492965698242 nb_pixel_total : 103658 time to create 1 rle with old method : 0.1193702220916748 time for calcul the mask position with numpy : 0.021567106246948242 nb_pixel_total : 26156 time to create 1 rle with old method : 0.03076958656311035 time for calcul the mask position with numpy : 0.022216796875 nb_pixel_total : 43241 time to create 1 rle with old method : 0.04713320732116699 time for calcul the mask position with numpy : 0.0207211971282959 nb_pixel_total : 6046 time to create 1 rle with old method : 0.006700992584228516 time for calcul the mask position with numpy : 0.021022558212280273 nb_pixel_total : 856 time to create 1 rle with old method : 0.0009987354278564453 time for calcul the mask position with numpy : 0.02154850959777832 nb_pixel_total : 153844 time to create 1 rle with new method : 0.6570141315460205 time for calcul the mask position with numpy : 0.021106243133544922 nb_pixel_total : 1928 time to create 1 rle with old method : 0.002413511276245117 time for calcul the mask position with numpy : 0.02115941047668457 nb_pixel_total : 42769 time to create 1 rle with old method : 0.048125505447387695 time for calcul the mask position with numpy : 0.021109342575073242 nb_pixel_total : 2533 time to create 1 rle with old method : 0.003295421600341797 time for calcul the mask position with numpy : 0.021436691284179688 nb_pixel_total : 65818 time to create 1 rle with old method : 0.07190418243408203 time for calcul the mask position with numpy : 0.02290654182434082 nb_pixel_total : 30807 time to create 1 rle with old method : 0.0338292121887207 time for calcul the mask position with numpy : 0.021277427673339844 nb_pixel_total : 25744 time to create 1 rle with old method : 0.028151988983154297 time for calcul the mask position with numpy : 0.0215609073638916 nb_pixel_total : 36254 time to create 1 rle with old method : 0.04167628288269043 time for calcul the mask position with numpy : 0.023712873458862305 nb_pixel_total : 3757 time to create 1 rle with old method : 0.004610300064086914 time for calcul the mask position with numpy : 0.02263641357421875 nb_pixel_total : 15311 time to create 1 rle with old method : 0.018887042999267578 time for calcul the mask position with numpy : 0.02351069450378418 nb_pixel_total : 2630 time to create 1 rle with old method : 0.003279447555541992 create new chi : 2.825605630874634 time to delete rle : 0.0012989044189453125 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 10494 TO DO : save crop sub photo not yet done ! save time : 0.8367459774017334 nb_obj : 18 nb_hashtags : 6 time to prepare the origin masks : 12.399264574050903 time for calcul the mask position with numpy : 0.3138084411621094 nb_pixel_total : 5834475 time to create 1 rle with new method : 0.6236257553100586 time for calcul the mask position with numpy : 0.035768985748291016 nb_pixel_total : 3605 time to create 1 rle with old method : 0.0039370059967041016 time for calcul the mask position with numpy : 0.02539682388305664 nb_pixel_total : 2702 time to create 1 rle with old method : 0.0032486915588378906 time for calcul the mask position with numpy : 0.039433956146240234 nb_pixel_total : 512832 time to create 1 rle with new method : 0.6319489479064941 time for calcul the mask position with numpy : 0.02494525909423828 nb_pixel_total : 1915 time to create 1 rle with old method : 0.002465486526489258 time for calcul the mask position with numpy : 0.02720928192138672 nb_pixel_total : 62036 time to create 1 rle with old method : 0.06688594818115234 time for calcul the mask position with numpy : 0.03546023368835449 nb_pixel_total : 12632 time to create 1 rle with old method : 0.013990640640258789 time for calcul the mask position with numpy : 0.03822135925292969 nb_pixel_total : 11735 time to create 1 rle with old method : 0.013247489929199219 time for calcul the mask position with numpy : 0.036522865295410156 nb_pixel_total : 5594 time to create 1 rle with old method : 0.006290912628173828 time for calcul the mask position with numpy : 0.036585092544555664 nb_pixel_total : 236050 time to create 1 rle with new method : 0.9435684680938721 time for calcul the mask position with numpy : 0.034857749938964844 nb_pixel_total : 5647 time to create 1 rle with old method : 0.006410837173461914 time for calcul the mask position with numpy : 0.03459763526916504 nb_pixel_total : 1526 time to create 1 rle with old method : 0.002129077911376953 time for calcul the mask position with numpy : 0.03623032569885254 nb_pixel_total : 9584 time to create 1 rle with old method : 0.012464284896850586 time for calcul the mask position with numpy : 0.032401084899902344 nb_pixel_total : 211555 time to create 1 rle with new method : 0.6496615409851074 time for calcul the mask position with numpy : 0.028326034545898438 nb_pixel_total : 4686 time to create 1 rle with old method : 0.005838155746459961 time for calcul the mask position with numpy : 0.021483421325683594 nb_pixel_total : 9318 time to create 1 rle with old method : 0.0106353759765625 time for calcul the mask position with numpy : 0.021727323532104492 nb_pixel_total : 1174 time to create 1 rle with old method : 0.0013580322265625 time for calcul the mask position with numpy : 0.02187943458557129 nb_pixel_total : 60695 time to create 1 rle with old method : 0.07694673538208008 time for calcul the mask position with numpy : 0.023501157760620117 nb_pixel_total : 62479 time to create 1 rle with old method : 0.06819367408752441 create new chi : 4.1265037059783936 time to delete rle : 0.001447439193725586 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 10265 TO DO : save crop sub photo not yet done ! save time : 0.8768205642700195 nb_obj : 23 nb_hashtags : 7 time to prepare the origin masks : 12.4040846824646 time for calcul the mask position with numpy : 0.29888343811035156 nb_pixel_total : 5428267 time to create 1 rle with new method : 0.7914295196533203 time for calcul the mask position with numpy : 0.02114558219909668 nb_pixel_total : 3968 time to create 1 rle with old method : 0.0048961639404296875 time for calcul the mask position with numpy : 0.021352529525756836 nb_pixel_total : 64744 time to create 1 rle with old method : 0.0725700855255127 time for calcul the mask position with numpy : 0.024243831634521484 nb_pixel_total : 307718 time to create 1 rle with new method : 0.479947566986084 time for calcul the mask position with numpy : 0.03676605224609375 nb_pixel_total : 3765 time to create 1 rle with old method : 0.0043218135833740234 time for calcul the mask position with numpy : 0.04000043869018555 nb_pixel_total : 104278 time to create 1 rle with old method : 0.11698198318481445 time for calcul the mask position with numpy : 0.03746747970581055 nb_pixel_total : 376597 time to create 1 rle with new method : 0.48806071281433105 time for calcul the mask position with numpy : 0.03417396545410156 nb_pixel_total : 37181 time to create 1 rle with old method : 0.039989471435546875 time for calcul the mask position with numpy : 0.03593325614929199 nb_pixel_total : 152839 time to create 1 rle with new method : 0.478395938873291 time for calcul the mask position with numpy : 0.034806013107299805 nb_pixel_total : 1429 time to create 1 rle with old method : 0.001875162124633789 time for calcul the mask position with numpy : 0.034331321716308594 nb_pixel_total : 3090 time to create 1 rle with old method : 0.003509521484375 time for calcul the mask position with numpy : 0.034688711166381836 nb_pixel_total : 20876 time to create 1 rle with old method : 0.022362709045410156 time for calcul the mask position with numpy : 0.04190874099731445 nb_pixel_total : 268853 time to create 1 rle with new method : 0.8703057765960693 time for calcul the mask position with numpy : 0.04306817054748535 nb_pixel_total : 49243 time to create 1 rle with old method : 0.05202174186706543 time for calcul the mask position with numpy : 0.03430676460266113 nb_pixel_total : 4045 time to create 1 rle with old method : 0.004652976989746094 time for calcul the mask position with numpy : 0.035558462142944336 nb_pixel_total : 11989 time to create 1 rle with old method : 0.012892961502075195 time for calcul the mask position with numpy : 0.034485816955566406 nb_pixel_total : 3745 time to create 1 rle with old method : 0.004148244857788086 time for calcul the mask position with numpy : 0.03440999984741211 nb_pixel_total : 34351 time to create 1 rle with old method : 0.036846160888671875 time for calcul the mask position with numpy : 0.032926321029663086 nb_pixel_total : 2177 time to create 1 rle with old method : 0.0024688243865966797 time for calcul the mask position with numpy : 0.03345942497253418 nb_pixel_total : 61744 time to create 1 rle with old method : 0.08549308776855469 time for calcul the mask position with numpy : 0.037001848220825195 nb_pixel_total : 53718 time to create 1 rle with old method : 0.05719494819641113 time for calcul the mask position with numpy : 0.034004926681518555 nb_pixel_total : 1571 time to create 1 rle with old method : 0.0017769336700439453 time for calcul the mask position with numpy : 0.03767824172973633 nb_pixel_total : 52700 time to create 1 rle with old method : 0.056212663650512695 time for calcul the mask position with numpy : 0.034653663635253906 nb_pixel_total : 1352 time to create 1 rle with old method : 0.0015978813171386719 create new chi : 5.008893966674805 time to delete rle : 0.0021409988403320312 batch 1 Loaded 24 chid ids of type : 3726 Number RLEs to save : 17405 TO DO : save crop sub photo not yet done ! save time : 3.6606671810150146 nb_obj : 12 nb_hashtags : 4 time to prepare the origin masks : 8.292031526565552 time for calcul the mask position with numpy : 0.4710671901702881 nb_pixel_total : 5782352 time to create 1 rle with new method : 0.5918169021606445 time for calcul the mask position with numpy : 0.020940303802490234 nb_pixel_total : 33946 time to create 1 rle with old method : 0.03731060028076172 time for calcul the mask position with numpy : 0.0220186710357666 nb_pixel_total : 35617 time to create 1 rle with old method : 0.04981660842895508 time for calcul the mask position with numpy : 0.028408288955688477 nb_pixel_total : 926 time to create 1 rle with old method : 0.0016484260559082031 time for calcul the mask position with numpy : 0.0673975944519043 nb_pixel_total : 8123 time to create 1 rle with old method : 0.012408971786499023 time for calcul the mask position with numpy : 0.02738213539123535 nb_pixel_total : 90014 time to create 1 rle with old method : 0.13482189178466797 time for calcul the mask position with numpy : 0.023823976516723633 nb_pixel_total : 32895 time to create 1 rle with old method : 0.04711294174194336 time for calcul the mask position with numpy : 0.02890467643737793 nb_pixel_total : 335123 time to create 1 rle with new method : 0.7232632637023926 time for calcul the mask position with numpy : 0.022806167602539062 nb_pixel_total : 40924 time to create 1 rle with old method : 0.042897701263427734 time for calcul the mask position with numpy : 0.022413015365600586 nb_pixel_total : 181244 time to create 1 rle with new method : 0.3102107048034668 time for calcul the mask position with numpy : 0.022437334060668945 nb_pixel_total : 251309 time to create 1 rle with new method : 0.5600459575653076 time for calcul the mask position with numpy : 0.03739762306213379 nb_pixel_total : 6189 time to create 1 rle with old method : 0.006654977798461914 time for calcul the mask position with numpy : 0.034368038177490234 nb_pixel_total : 251578 time to create 1 rle with new method : 0.3861682415008545 create new chi : 3.86356782913208 time to delete rle : 0.0010929107666015625 batch 1 Loaded 13 chid ids of type : 3726 Number RLEs to save : 11030 TO DO : save crop sub photo not yet done ! save time : 2.17858624458313 nb_obj : 15 nb_hashtags : 6 time to prepare the origin masks : 13.108901739120483 time for calcul the mask position with numpy : 0.21297860145568848 nb_pixel_total : 5720871 time to create 1 rle with new method : 0.32566404342651367 time for calcul the mask position with numpy : 0.0226747989654541 nb_pixel_total : 87729 time to create 1 rle with old method : 0.09731864929199219 time for calcul the mask position with numpy : 0.0353083610534668 nb_pixel_total : 87733 time to create 1 rle with old method : 0.09557771682739258 time for calcul the mask position with numpy : 0.03588390350341797 nb_pixel_total : 18792 time to create 1 rle with old method : 0.020581483840942383 time for calcul the mask position with numpy : 0.03307700157165527 nb_pixel_total : 63561 time to create 1 rle with old method : 0.06818962097167969 time for calcul the mask position with numpy : 0.021818876266479492 nb_pixel_total : 7475 time to create 1 rle with old method : 0.008377313613891602 time for calcul the mask position with numpy : 0.02182459831237793 nb_pixel_total : 9822 time to create 1 rle with old method : 0.010743141174316406 time for calcul the mask position with numpy : 0.022806406021118164 nb_pixel_total : 12146 time to create 1 rle with old method : 0.013320684432983398 time for calcul the mask position with numpy : 0.0251312255859375 nb_pixel_total : 368505 time to create 1 rle with new method : 0.4231526851654053 time for calcul the mask position with numpy : 0.021597623825073242 nb_pixel_total : 42348 time to create 1 rle with old method : 0.04541921615600586 time for calcul the mask position with numpy : 0.02124953269958496 nb_pixel_total : 17263 time to create 1 rle with old method : 0.019398212432861328 time for calcul the mask position with numpy : 0.020716428756713867 nb_pixel_total : 12733 time to create 1 rle with old method : 0.014322519302368164 time for calcul the mask position with numpy : 0.023645639419555664 nb_pixel_total : 427462 time to create 1 rle with new method : 0.34074997901916504 time for calcul the mask position with numpy : 0.02328348159790039 nb_pixel_total : 1215 time to create 1 rle with old method : 0.002069234848022461 time for calcul the mask position with numpy : 0.025209426879882812 nb_pixel_total : 2424 time to create 1 rle with old method : 0.004033088684082031 time for calcul the mask position with numpy : 0.025190353393554688 nb_pixel_total : 170161 time to create 1 rle with new method : 0.3412480354309082 create new chi : 2.53224515914917 time to delete rle : 0.0010492801666259766 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 12157 TO DO : save crop sub photo not yet done ! save time : 2.996689796447754 nb_obj : 5 nb_hashtags : 5 time to prepare the origin masks : 4.359598159790039 time for calcul the mask position with numpy : 0.03370380401611328 nb_pixel_total : 1954853 time to create 1 rle with new method : 0.2793412208557129 time for calcul the mask position with numpy : 0.02200913429260254 nb_pixel_total : 574 time to create 1 rle with old method : 0.0008990764617919922 time for calcul the mask position with numpy : 0.020397663116455078 nb_pixel_total : 25671 time to create 1 rle with old method : 0.02737116813659668 time for calcul the mask position with numpy : 0.020520448684692383 nb_pixel_total : 38694 time to create 1 rle with old method : 0.041129350662231445 time for calcul the mask position with numpy : 0.023098468780517578 nb_pixel_total : 109924 time to create 1 rle with old method : 0.1178290843963623 time for calcul the mask position with numpy : 0.15436387062072754 nb_pixel_total : 4920524 time to create 1 rle with new method : 0.29449009895324707 create new chi : 1.08766770362854 time to delete rle : 0.0008389949798583984 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 8396 TO DO : save crop sub photo not yet done ! save time : 3.261762857437134 map_output_result : {1335352207: (0.0, 'Should be the crop_list due to order', 0.0), 1335352206: (0.0, 'Should be the crop_list due to order', 0.0), 1335352204: (0.0, 'Should be the crop_list due to order', 0.0), 1335352200: (0.0, 'Should be the crop_list due to order', 0.0), 1335350972: (0.0, 'Should be the crop_list due to order', 0.0), 1335350969: (0.0, 'Should be the crop_list due to order', 0.0), 1335350967: (0.0, 'Should be the crop_list due to order', 0.0), 1335348492: (0.0, 'Should be the crop_list due to order', 0.0), 1335348488: (0.0, 'Should be the crop_list due to order', 0.0), 1335348485: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1335352207, 1335352206, 1335352204, 1335352200, 1335350972, 1335350969, 1335350967, 1335348492, 1335348488, 1335348485] Looping around the photos to save general results len do output : 10 /1335352207.Didn't retrieve data . /1335352206.Didn't retrieve data . /1335352204.Didn't retrieve data . /1335352200.Didn't retrieve data . /1335350972.Didn't retrieve data . /1335350969.Didn't retrieve data . /1335350967.Didn't retrieve data . /1335348492.Didn't retrieve data . /1335348488.Didn't retrieve data . /1335348485.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352207', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352206', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352204', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352200', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350972', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350969', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350967', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348492', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348488', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348485', None, None, None, None, None, '2561530') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.15533137321472168 save_final save missing photos in datou_result : time spend for datou_step_exec : 156.3582739830017 time spend to save output : 0.15770411491394043 total time spend for step 5 : 156.51597809791565 step6:crop_condition Fri Feb 7 06:35:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 144 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738906562_2078403 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.4090993404388428 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738906571_2078403 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.4118947982788086 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! About to upload 9 photos upload in portfolio : 4869462 init cache_photo without model_param we have 9 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738906577_2078403 we have uploaded 9 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.8918190002441406 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738906581_2078403 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0436913967132568 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738906584_2078403 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.3122217655181885 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738906588_2078403 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0896999835968018 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1738906591_2078403 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.5881729125976562 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1335352207, 1335352206, 1335352204, 1335352200, 1335350972, 1335350969, 1335350967, 1335348492, 1335348488, 1335348485] Looping around the photos to save general results len do output : 30 /1335412365Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412377Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412417Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412419Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412422Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412426Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412429Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412432Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412435Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412439Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412441Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412442Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412443Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412444Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412445Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412446Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412447Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412448Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1335412449Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352207', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352206', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352204', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352200', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350972', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350969', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350967', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348492', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348488', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348485', None, None, None, None, None, '2561530') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 100 time used for this insertion : 0.035006046295166016 save_final save missing photos in datou_result : time spend for datou_step_exec : 40.42873740196228 time spend to save output : 0.10944676399230957 total time spend for step 6 : 40.53818416595459 step7:ventilate_hashtags_in_portfolio Fri Feb 7 06:36:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20321739 get user id for portfolio 20321739 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20321739 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','autre_refus','Carton_brun','cartonnette','mal_croppe','metal','papier','flou','Carton_gris','Teint_Dans_La_Masse','kraft','plastique')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20321739 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','autre_refus','Carton_brun','cartonnette','mal_croppe','metal','papier','flou','Carton_gris','Teint_Dans_La_Masse','kraft','plastique')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20321739 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('environnement','autre_refus','Carton_brun','cartonnette','mal_croppe','metal','papier','flou','Carton_gris','Teint_Dans_La_Masse','kraft','plastique')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20321947,20321948,20321949,20321950,20321951,20321953,20321954,20321955,20321956,20321957,20321958,20321959?tags=environnement,autre_refus,Carton_brun,cartonnette,mal_croppe,metal,papier,flou,Carton_gris,Teint_Dans_La_Masse,kraft,plastique Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1335352207, 1335352206, 1335352204, 1335352200, 1335350972, 1335350969, 1335350967, 1335348492, 1335348488, 1335348485] Looping around the photos to save general results len do output : 1 /20321739. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352207', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352206', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352204', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352200', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350972', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350969', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350967', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348492', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348488', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348485', None, None, None, None, None, '2561530') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.4497194290161133 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.0161707401275635 time spend to save output : 0.45005178451538086 total time spend for step 7 : 2.4662225246429443 step8:final Fri Feb 7 06:36:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1335352207: ('0.08940862040233692',), 1335352206: ('0.08940862040233692',), 1335352204: ('0.08940862040233692',), 1335352200: ('0.08940862040233692',), 1335350972: ('0.08940862040233692',), 1335350969: ('0.08940862040233692',), 1335350967: ('0.08940862040233692',), 1335348492: ('0.08940862040233692',), 1335348488: ('0.08940862040233692',), 1335348485: ('0.08940862040233692',)} new output for save of step final : {1335352207: ('0.08940862040233692',), 1335352206: ('0.08940862040233692',), 1335352204: ('0.08940862040233692',), 1335352200: ('0.08940862040233692',), 1335350972: ('0.08940862040233692',), 1335350969: ('0.08940862040233692',), 1335350967: ('0.08940862040233692',), 1335348492: ('0.08940862040233692',), 1335348488: ('0.08940862040233692',), 1335348485: ('0.08940862040233692',)} [1335352207, 1335352206, 1335352204, 1335352200, 1335350972, 1335350969, 1335350967, 1335348492, 1335348488, 1335348485] Looping around the photos to save general results len do output : 10 /1335352207.Didn't retrieve data . /1335352206.Didn't retrieve data . /1335352204.Didn't retrieve data . /1335352200.Didn't retrieve data . /1335350972.Didn't retrieve data . /1335350969.Didn't retrieve data . /1335350967.Didn't retrieve data . /1335348492.Didn't retrieve data . /1335348488.Didn't retrieve data . /1335348485.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352207', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352206', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352204', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352200', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350972', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350969', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350967', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348492', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348488', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348485', None, None, None, None, None, '2561530') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01910257339477539 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.6366417407989502 time spend to save output : 0.02132129669189453 total time spend for step 8 : 0.6579630374908447 step9:velours_tree Fri Feb 7 06:36:42 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.36228346824645996 time spend to save output : 4.506111145019531e-05 total time spend for step 9 : 0.36232852935791016 step10:send_mail_cod Fri Feb 7 06:36:43 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20321739_07-02-2025_06_36_43.pdf 20321948 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203219481738906603 20321950 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203219501738906605 20321951 imagette203219511738906606 20321953 imagette203219531738906606 20321955 imagette203219551738906606 20321958 change filename to text .change filename to text .change filename to text .change filename to text .imagette203219581738906606 20321959 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203219591738906606 20321949 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203219491738906607 20321956 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203219561738906608 20321957 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette203219571738906610 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20321739 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20321947,20321948,20321949,20321950,20321951,20321953,20321954,20321955,20321956,20321957,20321958,20321959?tags=environnement,autre_refus,Carton_brun,cartonnette,mal_croppe,metal,papier,flou,Carton_gris,Teint_Dans_La_Masse,kraft,plastique your option no_mail is active, we will not send the real mail to your client args[1335352207] : ('0.08940862040233692',) no score found for photo 1335352207 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335352206] : ('0.08940862040233692',) no score found for photo 1335352206 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335352204] : ('0.08940862040233692',) no score found for photo 1335352204 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335352200] : ('0.08940862040233692',) no score found for photo 1335352200 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335350972] : ('0.08940862040233692',) no score found for photo 1335350972 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335350969] : ('0.08940862040233692',) no score found for photo 1335350969 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335350967] : ('0.08940862040233692',) no score found for photo 1335350967 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335348492] : ('0.08940862040233692',) no score found for photo 1335348492 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335348488] : ('0.08940862040233692',) no score found for photo 1335348488 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com args[1335348485] : ('0.08940862040233692',) no score found for photo 1335348485 apple ('0.08940862040233692',) We are sending mail with results at report@fotonower.com refus_total : 0.08940862040233692 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20321739 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1335191696,1335352200,1335350972,1335350967,1335348492,1335348488,1335193060,1335193055,1335193052,1335193049,1335193011,1335193003,1335192939,1335192936,1335192933,1335192926,1335191693,1335191945,1335192938,1335191988) Found this number of photos: 20 begin to download photo : 1335191696 begin to download photo : 1335348488 begin to download photo : 1335193011 begin to download photo : 1335192926 download finish for photo 1335348488 begin to download photo : 1335193060 download finish for photo 1335192926 begin to download photo : 1335191693 download finish for photo 1335191696 begin to download photo : 1335352200 download finish for photo 1335193060 begin to download photo : 1335193055 download finish for photo 1335193011 begin to download photo : 1335193003 download finish for photo 1335352200 begin to download photo : 1335350972 download finish for photo 1335191693 begin to download photo : 1335191945 download finish for photo 1335193055 begin to download photo : 1335193052 download finish for photo 1335350972 begin to download photo : 1335350967 download finish for photo 1335193003 begin to download photo : 1335192939 download finish for photo 1335350967 begin to download photo : 1335348492 download finish for photo 1335191945 begin to download photo : 1335192938 download finish for photo 1335193052 begin to download photo : 1335193049 download finish for photo 1335192939 begin to download photo : 1335192936 download finish for photo 1335348492 download finish for photo 1335192938 begin to download photo : 1335191988 download finish for photo 1335192936 begin to download photo : 1335192933 download finish for photo 1335193049 download finish for photo 1335191988 download finish for photo 1335192933 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20321739_07-02-2025_06_36_43.pdf results_Auto_P20321739_07-02-2025_06_36_43.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20321739_07-02-2025_06_36_43.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20321739','results_Auto_P20321739_07-02-2025_06_36_43.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20321739_07-02-2025_06_36_43.pdf','pdf','','1.45','0.08940862040233692') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1335352207, 1335352206, 1335352204, 1335352200, 1335350972, 1335350969, 1335350967, 1335348492, 1335348488, 1335348485] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352207', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352206', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352204', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352200', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350972', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350969', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350967', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348492', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348488', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348485', None, None, None, None, None, '2561530') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.017058849334716797 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.593128204345703 time spend to save output : 0.01731419563293457 total time spend for step 10 : 14.610442399978638 step11:split_time_score Fri Feb 7 06:36:57 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 37),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 06022025 20321739 Nombre de photos uploadées : 37 / 23040 (0%) 06022025 20321739 Nombre de photos taguées (types de déchets): 0 / 37 (0%) 06022025 20321739 Nombre de photos taguées (volume) : 0 / 37 (0%) elapsed_time : load_data_split_time_score 5.0067901611328125e-06 elapsed_time : order_list_meta_photo_and_scores 2.288818359375e-05 ????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.003385782241821289 elapsed_time : insert_dashboard_record_day_entry 0.023862361907958984 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.21863425925925936 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314812_07-02-2025_02_52_49.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314812 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314812 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314813 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314814 order by id desc limit 1 Qualite : 0.5683061661597331 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314815_07-02-2025_06_09_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314815 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314815 AND mptpi.`type`=4200 To do Qualite : 0.09494094623872477 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314816_07-02-2025_06_21_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314816 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314816 AND mptpi.`type`=4207 To do Qualite : 0.06847575159241935 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314817_07-02-2025_05_20_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314817 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314817 AND mptpi.`type`=4207 To do Qualite : 0.05500822396038016 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314818_07-02-2025_05_06_55.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314818 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314818 AND mptpi.`type`=4207 To do Qualite : 0.07107308905535348 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314819_07-02-2025_06_01_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314819 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314819 AND mptpi.`type`=4207 To do Qualite : 0.052388112823274276 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314820_07-02-2025_04_29_38.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314820 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314820 AND mptpi.`type`=4207 To do Qualite : 0.3475520682200876 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314821_07-02-2025_03_08_37.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314821 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314821 AND mptpi.`type`=4200 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321707 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321708 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321709 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321710 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321711 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321736 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321737 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321738 order by id desc limit 1 Qualite : 0.08940862040233692 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20321739_07-02-2025_06_36_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321739 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20321739 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321740 order by id desc limit 1 Qualite : 0.07606678211572258 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314829_07-02-2025_03_59_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314829 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314829 AND mptpi.`type`=4207 To do Qualite : 0.10086758606120957 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314830_07-02-2025_04_18_00.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314830 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314830 AND mptpi.`type`=4207 To do Qualite : 0.06528518323435197 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314831_07-02-2025_03_09_05.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314831 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314831 AND mptpi.`type`=4207 To do Qualite : 0.08062900675827052 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314832_07-02-2025_03_50_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314832 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314832 AND mptpi.`type`=4207 To do Qualite : 0.053841303219029274 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314833_07-02-2025_02_40_39.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314833 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314833 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314834 order by id desc limit 1 Qualite : 0.22870597929691197 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314835_07-02-2025_02_26_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314835 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314835 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321741 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321742 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321743 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321744 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321745 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321746 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321748 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321749 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321750 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321752 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321753 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314843 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20321754 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314845 order by id desc limit 1 Qualite : 0.08977715635017561 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314846_07-02-2025_01_55_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314846 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314846 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314847 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314848 order by id desc limit 1 Qualite : 0.0951918982153927 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314849_07-02-2025_03_54_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314849 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314849 AND mptpi.`type`=4207 To do Qualite : 0.048795321359665696 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314850_07-02-2025_03_49_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314850 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314850 AND mptpi.`type`=4207 To do Qualite : 0.0794257709299137 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314851_07-02-2025_02_06_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314851 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314851 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314852 order by id desc limit 1 Qualite : 0.08573655308919938 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314853_07-02-2025_01_44_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314853 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314853 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314854 order by id desc limit 1 Qualite : 0.2245652913943355 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314855_07-02-2025_01_50_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314855 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20314855 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314856 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20314857_07-02-2025_02_05_12.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20314857 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'06022025': {'nb_upload': 37, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1335352207, 1335352206, 1335352204, 1335352200, 1335350972, 1335350969, 1335350967, 1335348492, 1335348488, 1335348485] Looping around the photos to save general results len do output : 1 /20321739Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352207', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352206', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352204', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335352200', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350972', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350969', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335350967', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348492', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348488', None, None, None, None, None, '2561530') ('3459', None, None, None, None, None, None, None, '2561530') ('3459', '20321739', '1335348485', None, None, None, None, None, '2561530') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.041960716247558594 save_final save missing photos in datou_result : time spend for datou_step_exec : 27.816906690597534 time spend to save output : 0.042229413986206055 total time spend for step 11 : 27.85913610458374 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 162.74user 151.77system 7:04.33elapsed 74%CPU (0avgtext+0avgdata 3597908maxresident)k 937664inputs+167160outputs (32932major+13057702minor)pagefaults 0swaps