python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2684112 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 20>10 please execute split_portfolio.py -i 20905796 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2621865'] with mtr_portfolio_ids : ['20905796'] and first list_photo_ids : [] new path : /proc/2684112/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.947472095489502 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Fri Feb 28 03:12:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5525 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-28 03:12:31.952850: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-28 03:12:31.979208: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-28 03:12:31.981188: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f3130000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:12:31.981236: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-28 03:12:31.984763: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-28 03:12:32.207913: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x28feeb70 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-28 03:12:32.207976: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-28 03:12:32.209228: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:12:32.209682: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:12:32.213181: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:12:32.216164: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:12:32.216677: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:12:32.219403: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:12:32.220882: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:12:32.226308: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:12:32.227848: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:12:32.227942: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:12:32.228716: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:12:32.228734: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:12:32.228746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:12:32.230063: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5051 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-28 03:12:32.484382: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:12:32.484499: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:12:32.484531: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:12:32.484560: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:12:32.484588: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:12:32.484615: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:12:32.484642: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:12:32.484669: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:12:32.486227: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:12:32.487605: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-28 03:12:32.487655: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-28 03:12:32.487681: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:12:32.487704: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-28 03:12:32.487727: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-28 03:12:32.487750: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-28 03:12:32.487772: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-28 03:12:32.487795: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-28 03:12:32.488778: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-28 03:12:32.488805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-28 03:12:32.488814: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-28 03:12:32.488822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-28 03:12:32.489848: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5051 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-28 03:12:42.849003: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-28 03:12:43.031830: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 7 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 15 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.07453 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 19 Detection mask done ! Trying to reset tf kernel 2684725 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 236 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5525 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.3088042736053467 nb_pixel_total : 3170715 time to create 1 rle with new method : 0.30913209915161133 length of segment : 4528 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 2858 time to create 1 rle with old method : 0.003438711166381836 length of segment : 83 time for calcul the mask position with numpy : 0.0009708404541015625 nb_pixel_total : 54382 time to create 1 rle with old method : 0.060925960540771484 length of segment : 456 time for calcul the mask position with numpy : 0.011445283889770508 nb_pixel_total : 533649 time to create 1 rle with new method : 0.022806167602539062 length of segment : 785 time for calcul the mask position with numpy : 0.009242057800292969 nb_pixel_total : 533297 time to create 1 rle with new method : 0.152909517288208 length of segment : 1447 time for calcul the mask position with numpy : 0.003656148910522461 nb_pixel_total : 268853 time to create 1 rle with new method : 0.009285926818847656 length of segment : 619 time for calcul the mask position with numpy : 0.0010409355163574219 nb_pixel_total : 58678 time to create 1 rle with old method : 0.06506705284118652 length of segment : 341 time for calcul the mask position with numpy : 0.00016427040100097656 nb_pixel_total : 4935 time to create 1 rle with old method : 0.005896091461181641 length of segment : 115 time for calcul the mask position with numpy : 0.0030062198638916016 nb_pixel_total : 100192 time to create 1 rle with old method : 0.11215877532958984 length of segment : 403 time for calcul the mask position with numpy : 0.0014197826385498047 nb_pixel_total : 65897 time to create 1 rle with old method : 0.07213473320007324 length of segment : 321 time for calcul the mask position with numpy : 0.001401662826538086 nb_pixel_total : 65091 time to create 1 rle with old method : 0.06910300254821777 length of segment : 360 time for calcul the mask position with numpy : 0.0017545223236083984 nb_pixel_total : 104658 time to create 1 rle with old method : 0.1144251823425293 length of segment : 385 time for calcul the mask position with numpy : 0.002403736114501953 nb_pixel_total : 97956 time to create 1 rle with old method : 0.10209417343139648 length of segment : 458 time for calcul the mask position with numpy : 0.0001811981201171875 nb_pixel_total : 6860 time to create 1 rle with old method : 0.0076329708099365234 length of segment : 86 time for calcul the mask position with numpy : 0.0033943653106689453 nb_pixel_total : 103600 time to create 1 rle with old method : 0.11332511901855469 length of segment : 598 time for calcul the mask position with numpy : 0.00036644935607910156 nb_pixel_total : 16865 time to create 1 rle with old method : 0.019195556640625 length of segment : 101 time for calcul the mask position with numpy : 0.001094818115234375 nb_pixel_total : 59286 time to create 1 rle with old method : 0.06243562698364258 length of segment : 431 time for calcul the mask position with numpy : 0.0019712448120117188 nb_pixel_total : 113338 time to create 1 rle with old method : 0.12247157096862793 length of segment : 375 time for calcul the mask position with numpy : 0.0009944438934326172 nb_pixel_total : 16625 time to create 1 rle with old method : 0.018529653549194336 length of segment : 164 time for calcul the mask position with numpy : 0.00012946128845214844 nb_pixel_total : 372 time to create 1 rle with old method : 0.0004372596740722656 length of segment : 32 time for calcul the mask position with numpy : 0.0007472038269042969 nb_pixel_total : 8658 time to create 1 rle with old method : 0.009559154510498047 length of segment : 204 time for calcul the mask position with numpy : 0.0002865791320800781 nb_pixel_total : 4024 time to create 1 rle with old method : 0.004579782485961914 length of segment : 84 time for calcul the mask position with numpy : 0.005954742431640625 nb_pixel_total : 119840 time to create 1 rle with old method : 0.12740302085876465 length of segment : 796 time for calcul the mask position with numpy : 0.0003542900085449219 nb_pixel_total : 11048 time to create 1 rle with old method : 0.011787652969360352 length of segment : 243 time for calcul the mask position with numpy : 0.0012218952178955078 nb_pixel_total : 43795 time to create 1 rle with old method : 0.04896402359008789 length of segment : 306 time for calcul the mask position with numpy : 0.00012540817260742188 nb_pixel_total : 855 time to create 1 rle with old method : 0.0011222362518310547 length of segment : 30 time for calcul the mask position with numpy : 0.0030307769775390625 nb_pixel_total : 84437 time to create 1 rle with old method : 0.09336352348327637 length of segment : 448 time for calcul the mask position with numpy : 0.0002620220184326172 nb_pixel_total : 3480 time to create 1 rle with old method : 0.00404667854309082 length of segment : 100 time for calcul the mask position with numpy : 0.0004825592041015625 nb_pixel_total : 15567 time to create 1 rle with old method : 0.017238855361938477 length of segment : 193 time for calcul the mask position with numpy : 0.0007567405700683594 nb_pixel_total : 27553 time to create 1 rle with old method : 0.029885053634643555 length of segment : 241 time for calcul the mask position with numpy : 0.0007092952728271484 nb_pixel_total : 29979 time to create 1 rle with old method : 0.03274989128112793 length of segment : 221 time for calcul the mask position with numpy : 0.0003654956817626953 nb_pixel_total : 7814 time to create 1 rle with old method : 0.008992195129394531 length of segment : 125 time for calcul the mask position with numpy : 0.0006575584411621094 nb_pixel_total : 27631 time to create 1 rle with old method : 0.03201103210449219 length of segment : 183 time for calcul the mask position with numpy : 0.0013880729675292969 nb_pixel_total : 56419 time to create 1 rle with old method : 0.06445980072021484 length of segment : 280 time for calcul the mask position with numpy : 0.0006508827209472656 nb_pixel_total : 21740 time to create 1 rle with old method : 0.025321006774902344 length of segment : 168 time for calcul the mask position with numpy : 0.0009906291961669922 nb_pixel_total : 55097 time to create 1 rle with old method : 0.06171560287475586 length of segment : 286 time for calcul the mask position with numpy : 0.0013580322265625 nb_pixel_total : 35295 time to create 1 rle with old method : 0.03989744186401367 length of segment : 285 time for calcul the mask position with numpy : 0.0014998912811279297 nb_pixel_total : 49374 time to create 1 rle with old method : 0.05352044105529785 length of segment : 357 time for calcul the mask position with numpy : 0.002280712127685547 nb_pixel_total : 105639 time to create 1 rle with old method : 0.11261725425720215 length of segment : 350 time for calcul the mask position with numpy : 0.0019168853759765625 nb_pixel_total : 80613 time to create 1 rle with old method : 0.0960845947265625 length of segment : 472 time for calcul the mask position with numpy : 0.0050258636474609375 nb_pixel_total : 229396 time to create 1 rle with new method : 0.009407520294189453 length of segment : 547 time for calcul the mask position with numpy : 0.0018475055694580078 nb_pixel_total : 84623 time to create 1 rle with old method : 0.09173321723937988 length of segment : 332 time for calcul the mask position with numpy : 0.0023958683013916016 nb_pixel_total : 68756 time to create 1 rle with old method : 0.08629012107849121 length of segment : 347 time for calcul the mask position with numpy : 0.0003883838653564453 nb_pixel_total : 12629 time to create 1 rle with old method : 0.014585256576538086 length of segment : 160 time for calcul the mask position with numpy : 0.0003197193145751953 nb_pixel_total : 6956 time to create 1 rle with old method : 0.008009672164916992 length of segment : 88 time for calcul the mask position with numpy : 0.0002505779266357422 nb_pixel_total : 2341 time to create 1 rle with old method : 0.0027730464935302734 length of segment : 98 time for calcul the mask position with numpy : 0.0005748271942138672 nb_pixel_total : 35834 time to create 1 rle with old method : 0.03934836387634277 length of segment : 349 time for calcul the mask position with numpy : 0.005415201187133789 nb_pixel_total : 240325 time to create 1 rle with new method : 0.009641408920288086 length of segment : 540 time for calcul the mask position with numpy : 0.00018668174743652344 nb_pixel_total : 1109 time to create 1 rle with old method : 0.0013992786407470703 length of segment : 72 time for calcul the mask position with numpy : 0.0003256797790527344 nb_pixel_total : 10522 time to create 1 rle with old method : 0.012590885162353516 length of segment : 156 time for calcul the mask position with numpy : 0.000141143798828125 nb_pixel_total : 2283 time to create 1 rle with old method : 0.002798795700073242 length of segment : 41 time for calcul the mask position with numpy : 0.00040078163146972656 nb_pixel_total : 6422 time to create 1 rle with old method : 0.009167909622192383 length of segment : 112 time for calcul the mask position with numpy : 0.00191497802734375 nb_pixel_total : 50904 time to create 1 rle with old method : 0.05565929412841797 length of segment : 420 time for calcul the mask position with numpy : 0.0016431808471679688 nb_pixel_total : 37724 time to create 1 rle with old method : 0.04375600814819336 length of segment : 404 time for calcul the mask position with numpy : 0.00024890899658203125 nb_pixel_total : 3925 time to create 1 rle with old method : 0.004675626754760742 length of segment : 88 time for calcul the mask position with numpy : 0.031167984008789062 nb_pixel_total : 1029271 time to create 1 rle with new method : 0.3246138095855713 length of segment : 1782 time for calcul the mask position with numpy : 0.0002474784851074219 nb_pixel_total : 4314 time to create 1 rle with old method : 0.005116701126098633 length of segment : 59 time for calcul the mask position with numpy : 0.0003952980041503906 nb_pixel_total : 9233 time to create 1 rle with old method : 0.010967493057250977 length of segment : 101 time for calcul the mask position with numpy : 0.003524303436279297 nb_pixel_total : 116464 time to create 1 rle with old method : 0.12824559211730957 length of segment : 635 time for calcul the mask position with numpy : 0.0034036636352539062 nb_pixel_total : 123630 time to create 1 rle with old method : 0.16029930114746094 length of segment : 635 time for calcul the mask position with numpy : 0.0012302398681640625 nb_pixel_total : 33590 time to create 1 rle with old method : 0.03851890563964844 length of segment : 249 time for calcul the mask position with numpy : 0.0016121864318847656 nb_pixel_total : 46889 time to create 1 rle with old method : 0.054388999938964844 length of segment : 446 time for calcul the mask position with numpy : 0.0004143714904785156 nb_pixel_total : 16081 time to create 1 rle with old method : 0.0188596248626709 length of segment : 177 time for calcul the mask position with numpy : 0.0030641555786132812 nb_pixel_total : 98161 time to create 1 rle with old method : 0.11798977851867676 length of segment : 800 time for calcul the mask position with numpy : 0.0010688304901123047 nb_pixel_total : 54609 time to create 1 rle with old method : 0.06165361404418945 length of segment : 188 time for calcul the mask position with numpy : 0.0007281303405761719 nb_pixel_total : 30966 time to create 1 rle with old method : 0.03570079803466797 length of segment : 282 time for calcul the mask position with numpy : 0.0020987987518310547 nb_pixel_total : 117659 time to create 1 rle with old method : 0.1332237720489502 length of segment : 619 time for calcul the mask position with numpy : 0.0008637905120849609 nb_pixel_total : 46258 time to create 1 rle with old method : 0.052625417709350586 length of segment : 444 time for calcul the mask position with numpy : 0.01282358169555664 nb_pixel_total : 529305 time to create 1 rle with new method : 0.18934869766235352 length of segment : 962 time for calcul the mask position with numpy : 0.00011682510375976562 nb_pixel_total : 637 time to create 1 rle with old method : 0.0008385181427001953 length of segment : 33 time for calcul the mask position with numpy : 0.0024747848510742188 nb_pixel_total : 86556 time to create 1 rle with old method : 0.09934306144714355 length of segment : 600 time for calcul the mask position with numpy : 0.0005679130554199219 nb_pixel_total : 15202 time to create 1 rle with old method : 0.017348766326904297 length of segment : 225 time for calcul the mask position with numpy : 0.0002052783966064453 nb_pixel_total : 4684 time to create 1 rle with old method : 0.005470991134643555 length of segment : 83 time for calcul the mask position with numpy : 0.0027196407318115234 nb_pixel_total : 108689 time to create 1 rle with old method : 0.12108421325683594 length of segment : 437 time for calcul the mask position with numpy : 0.0017731189727783203 nb_pixel_total : 30374 time to create 1 rle with old method : 0.03457951545715332 length of segment : 441 time for calcul the mask position with numpy : 0.0001995563507080078 nb_pixel_total : 1924 time to create 1 rle with old method : 0.002592802047729492 length of segment : 60 time for calcul the mask position with numpy : 0.0029821395874023438 nb_pixel_total : 115814 time to create 1 rle with old method : 0.12915277481079102 length of segment : 398 time for calcul the mask position with numpy : 0.0008149147033691406 nb_pixel_total : 13570 time to create 1 rle with old method : 0.01521754264831543 length of segment : 230 time for calcul the mask position with numpy : 0.0004115104675292969 nb_pixel_total : 9624 time to create 1 rle with old method : 0.010859251022338867 length of segment : 146 time for calcul the mask position with numpy : 0.0029370784759521484 nb_pixel_total : 169315 time to create 1 rle with new method : 0.006212711334228516 length of segment : 776 time for calcul the mask position with numpy : 0.0001666545867919922 nb_pixel_total : 1423 time to create 1 rle with old method : 0.0017971992492675781 length of segment : 62 time for calcul the mask position with numpy : 0.0006620883941650391 nb_pixel_total : 17044 time to create 1 rle with old method : 0.020097017288208008 length of segment : 167 time for calcul the mask position with numpy : 0.002068758010864258 nb_pixel_total : 113653 time to create 1 rle with old method : 0.1284022331237793 length of segment : 447 time for calcul the mask position with numpy : 0.0016169548034667969 nb_pixel_total : 23188 time to create 1 rle with old method : 0.03720736503601074 length of segment : 421 time for calcul the mask position with numpy : 0.00012373924255371094 nb_pixel_total : 2146 time to create 1 rle with old method : 0.002705812454223633 length of segment : 46 time for calcul the mask position with numpy : 0.00026035308837890625 nb_pixel_total : 3535 time to create 1 rle with old method : 0.004350423812866211 length of segment : 89 time for calcul the mask position with numpy : 0.0025925636291503906 nb_pixel_total : 70515 time to create 1 rle with old method : 0.07881450653076172 length of segment : 397 time for calcul the mask position with numpy : 0.0025157928466796875 nb_pixel_total : 80411 time to create 1 rle with old method : 0.08945703506469727 length of segment : 546 time for calcul the mask position with numpy : 0.006606101989746094 nb_pixel_total : 293675 time to create 1 rle with new method : 0.021269798278808594 length of segment : 733 time for calcul the mask position with numpy : 0.0011162757873535156 nb_pixel_total : 49517 time to create 1 rle with old method : 0.054807186126708984 length of segment : 335 time for calcul the mask position with numpy : 0.0027675628662109375 nb_pixel_total : 108736 time to create 1 rle with old method : 0.12850236892700195 length of segment : 506 time for calcul the mask position with numpy : 0.0005002021789550781 nb_pixel_total : 10743 time to create 1 rle with old method : 0.012799263000488281 length of segment : 154 time for calcul the mask position with numpy : 0.0002720355987548828 nb_pixel_total : 13255 time to create 1 rle with old method : 0.015568017959594727 length of segment : 177 time for calcul the mask position with numpy : 0.005742073059082031 nb_pixel_total : 233901 time to create 1 rle with new method : 0.008791208267211914 length of segment : 728 time for calcul the mask position with numpy : 0.0002815723419189453 nb_pixel_total : 4143 time to create 1 rle with old method : 0.0049076080322265625 length of segment : 109 time for calcul the mask position with numpy : 0.0007939338684082031 nb_pixel_total : 21691 time to create 1 rle with old method : 0.024669408798217773 length of segment : 184 time for calcul the mask position with numpy : 0.0006635189056396484 nb_pixel_total : 18127 time to create 1 rle with old method : 0.02091217041015625 length of segment : 207 time for calcul the mask position with numpy : 0.033616065979003906 nb_pixel_total : 938922 time to create 1 rle with new method : 0.22968101501464844 length of segment : 1470 time for calcul the mask position with numpy : 0.0037577152252197266 nb_pixel_total : 274655 time to create 1 rle with new method : 0.009175539016723633 length of segment : 1044 time for calcul the mask position with numpy : 0.0003407001495361328 nb_pixel_total : 11134 time to create 1 rle with old method : 0.013673543930053711 length of segment : 169 time for calcul the mask position with numpy : 0.0002231597900390625 nb_pixel_total : 5040 time to create 1 rle with old method : 0.005919694900512695 length of segment : 79 time for calcul the mask position with numpy : 0.0013530254364013672 nb_pixel_total : 36992 time to create 1 rle with old method : 0.04176640510559082 length of segment : 416 time for calcul the mask position with numpy : 0.0027201175689697266 nb_pixel_total : 101505 time to create 1 rle with old method : 0.10946989059448242 length of segment : 442 time for calcul the mask position with numpy : 0.00023555755615234375 nb_pixel_total : 8814 time to create 1 rle with old method : 0.010056257247924805 length of segment : 131 time for calcul the mask position with numpy : 0.0007157325744628906 nb_pixel_total : 19040 time to create 1 rle with old method : 0.022579669952392578 length of segment : 137 time for calcul the mask position with numpy : 0.00019240379333496094 nb_pixel_total : 2089 time to create 1 rle with old method : 0.002398252487182617 length of segment : 83 time for calcul the mask position with numpy : 0.0008175373077392578 nb_pixel_total : 22927 time to create 1 rle with old method : 0.024837732315063477 length of segment : 224 time for calcul the mask position with numpy : 0.00038170814514160156 nb_pixel_total : 9731 time to create 1 rle with old method : 0.0106964111328125 length of segment : 131 time for calcul the mask position with numpy : 0.004145622253417969 nb_pixel_total : 295823 time to create 1 rle with new method : 0.011840581893920898 length of segment : 841 time for calcul the mask position with numpy : 0.002836465835571289 nb_pixel_total : 145356 time to create 1 rle with old method : 0.16912388801574707 length of segment : 629 time for calcul the mask position with numpy : 0.0014190673828125 nb_pixel_total : 96257 time to create 1 rle with old method : 0.10833859443664551 length of segment : 562 time for calcul the mask position with numpy : 0.0053331851959228516 nb_pixel_total : 83877 time to create 1 rle with old method : 0.10039949417114258 length of segment : 672 time for calcul the mask position with numpy : 0.0003814697265625 nb_pixel_total : 11208 time to create 1 rle with old method : 0.01345205307006836 length of segment : 99 time for calcul the mask position with numpy : 0.00023794174194335938 nb_pixel_total : 4067 time to create 1 rle with old method : 0.004983663558959961 length of segment : 66 time for calcul the mask position with numpy : 0.00029921531677246094 nb_pixel_total : 5761 time to create 1 rle with old method : 0.006887674331665039 length of segment : 92 time for calcul the mask position with numpy : 0.0003566741943359375 nb_pixel_total : 4293 time to create 1 rle with old method : 0.005319833755493164 length of segment : 116 time for calcul the mask position with numpy : 0.0003142356872558594 nb_pixel_total : 8338 time to create 1 rle with old method : 0.00986337661743164 length of segment : 113 time for calcul the mask position with numpy : 0.0007522106170654297 nb_pixel_total : 18557 time to create 1 rle with old method : 0.02089524269104004 length of segment : 215 time for calcul the mask position with numpy : 0.003719806671142578 nb_pixel_total : 119680 time to create 1 rle with old method : 0.13244009017944336 length of segment : 438 time for calcul the mask position with numpy : 0.002180814743041992 nb_pixel_total : 54122 time to create 1 rle with old method : 0.06419253349304199 length of segment : 412 time for calcul the mask position with numpy : 0.02797245979309082 nb_pixel_total : 908938 time to create 1 rle with new method : 0.1334996223449707 length of segment : 1758 time for calcul the mask position with numpy : 0.0009996891021728516 nb_pixel_total : 37095 time to create 1 rle with old method : 0.040474891662597656 length of segment : 244 time for calcul the mask position with numpy : 0.0005428791046142578 nb_pixel_total : 13907 time to create 1 rle with old method : 0.01562976837158203 length of segment : 121 time for calcul the mask position with numpy : 0.0029668807983398438 nb_pixel_total : 209489 time to create 1 rle with new method : 0.009022712707519531 length of segment : 788 time for calcul the mask position with numpy : 0.00013446807861328125 nb_pixel_total : 908 time to create 1 rle with old method : 0.0011675357818603516 length of segment : 39 time for calcul the mask position with numpy : 0.0009996891021728516 nb_pixel_total : 29638 time to create 1 rle with old method : 0.03537940979003906 length of segment : 134 time for calcul the mask position with numpy : 0.0002684593200683594 nb_pixel_total : 5874 time to create 1 rle with old method : 0.006679058074951172 length of segment : 117 time for calcul the mask position with numpy : 0.002777099609375 nb_pixel_total : 202947 time to create 1 rle with new method : 0.007731199264526367 length of segment : 821 time for calcul the mask position with numpy : 0.00014901161193847656 nb_pixel_total : 3390 time to create 1 rle with old method : 0.004103183746337891 length of segment : 60 time for calcul the mask position with numpy : 0.002846956253051758 nb_pixel_total : 105807 time to create 1 rle with old method : 0.11731505393981934 length of segment : 399 time for calcul the mask position with numpy : 0.0005421638488769531 nb_pixel_total : 10256 time to create 1 rle with old method : 0.01195979118347168 length of segment : 132 time for calcul the mask position with numpy : 0.0014307498931884766 nb_pixel_total : 63672 time to create 1 rle with old method : 0.06923222541809082 length of segment : 509 time for calcul the mask position with numpy : 0.001384735107421875 nb_pixel_total : 54737 time to create 1 rle with old method : 0.06063556671142578 length of segment : 257 time for calcul the mask position with numpy : 0.0012416839599609375 nb_pixel_total : 48844 time to create 1 rle with old method : 0.05385422706604004 length of segment : 209 time for calcul the mask position with numpy : 0.00021529197692871094 nb_pixel_total : 2895 time to create 1 rle with old method : 0.0055506229400634766 length of segment : 72 time for calcul the mask position with numpy : 0.0010402202606201172 nb_pixel_total : 32506 time to create 1 rle with old method : 0.03585195541381836 length of segment : 238 time for calcul the mask position with numpy : 0.000797271728515625 nb_pixel_total : 30786 time to create 1 rle with old method : 0.034586191177368164 length of segment : 203 time for calcul the mask position with numpy : 0.003368854522705078 nb_pixel_total : 118168 time to create 1 rle with old method : 0.13875246047973633 length of segment : 515 time for calcul the mask position with numpy : 0.00333404541015625 nb_pixel_total : 127048 time to create 1 rle with old method : 0.14072918891906738 length of segment : 428 time for calcul the mask position with numpy : 0.0020813941955566406 nb_pixel_total : 35933 time to create 1 rle with old method : 0.041166067123413086 length of segment : 623 time for calcul the mask position with numpy : 0.0012269020080566406 nb_pixel_total : 78765 time to create 1 rle with old method : 0.08771824836730957 length of segment : 738 time for calcul the mask position with numpy : 0.00015497207641601562 nb_pixel_total : 1106 time to create 1 rle with old method : 0.0019545555114746094 length of segment : 47 time for calcul the mask position with numpy : 0.0015316009521484375 nb_pixel_total : 42052 time to create 1 rle with old method : 0.05236649513244629 length of segment : 408 time for calcul the mask position with numpy : 0.00057220458984375 nb_pixel_total : 25923 time to create 1 rle with old method : 0.029750823974609375 length of segment : 172 time for calcul the mask position with numpy : 0.0006794929504394531 nb_pixel_total : 26120 time to create 1 rle with old method : 0.0312345027923584 length of segment : 133 time for calcul the mask position with numpy : 0.0010385513305664062 nb_pixel_total : 32480 time to create 1 rle with old method : 0.03654837608337402 length of segment : 270 time spent for convertir_results : 17.88113498687744 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 151 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++Number RLEs to save : 55346 save missing photos in datou_result : time spend for datou_step_exec : 64.49952673912048 time spend to save output : 7.866978406906128 total time spend for step 1 : 72.36650514602661 step2:crop_condition Fri Feb 28 03:13:41 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 151 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 48 About to insert : list_path_to_insert length 48 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1340026538, 1340026522, 1340026513, 1340026473, 1340026456, 1340026452, 1340026110, 1340026109, 1340018842, 1340018836] Looping around the photos to save general results len do output : 92 /-3689358285Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358284Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358280Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358282Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449296Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358357Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358276Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358373Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358375Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358290Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358354Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358275Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358279Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358283Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358281Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449309Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449320Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449328Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358350Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358348Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358353Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358371Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358376Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358369Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358363Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358367Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449286Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358344Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689449287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3689358366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026538', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026522', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026513', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026473', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026456', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026452', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026110', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026109', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018842', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018836', None, None, None, None, None, '2621865') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 286 time used for this insertion : 0.03951454162597656 save_final save missing photos in datou_result : time spend for datou_step_exec : 37.42308950424194 time spend to save output : 0.0417022705078125 total time spend for step 2 : 37.464791774749756 step3:thcl Fri Feb 28 03:14:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.00522923469543457 time to convert the images to numpy array : 0.029878616333007812 time to import caffe and check if the image exist : 0.010407686233520508 time to convert the images to numpy array : 0.1131742000579834 time to import caffe and check if the image exist : 0.006103992462158203 time to convert the images to numpy array : 0.14615535736083984 time to import caffe and check if the image exist : 0.013500452041625977 time to convert the images to numpy array : 0.1814262866973877 time to import caffe and check if the image exist : 0.017817974090576172 time to convert the images to numpy array : 0.1795494556427002 time to import caffe and check if the image exist : 0.012259960174560547 time to convert the images to numpy array : 0.18698406219482422 time to import caffe and check if the image exist : 0.017324209213256836 time to convert the images to numpy array : 0.19515347480773926 time to import caffe and check if the image exist : 0.011450767517089844 time to convert the images to numpy array : 0.3260986804962158 time to import caffe and check if the image exist : 0.01517176628112793 time to convert the images to numpy array : 0.3305673599243164 time to import caffe and check if the image exist : 0.011362075805664062 time to convert the images to numpy array : 0.349621057510376 total time to convert the images to numpy array : 0.5892798900604248 list photo_ids error: [] list photo_ids correct : [-3689358360, -3689358366, -3689358301, -3689449286, -3689358324, -3689358330, -3689449299, -3689449289, -3689449318, -3689358344, -3689358326, -3689449287, -3689358285, -3689358284, -3689358280, -3689358282, -3689358309, -3689358316, -3689358331, -3689358327, -3689449301, -3689449296, -3689358371, -3689358376, -3689358364, -3689358374, -3689358368, -3689358369, -3689358363, -3689358362, -3689358367, -3689358302, -3689358354, -3689358275, -3689358288, -3689358295, -3689358299, -3689358289, -3689358291, -3689358279, -3689358283, -3689358297, -3689358370, -3689358373, -3689358375, -3689358290, -3689358308, -3689358332, -3689449304, -3689449295, -3689449300, -3689449317, -3689358336, -3689358339, -3689358325, -3689358338, -3689358337, -3689449309, -3689449297, -3689449292, -3689449308, -3689449319, -3689449326, -3689449320, -3689449328, -3689449327, -3689449322, -3689358350, -3689358358, -3689358348, -3689358356, -3689358353, -3689358281, -3689358303, -3689358306, -3689358307, -3689358313, -3689358311, -3689358315, -3689358314, -3689358312, -3689358320, -3689449294, -3689449291, -3689449311, -3689449316, -3689358357, -3689358276, -3689358319, -3689358333, -3689358340, -3689449325] number of photos to traite : 92 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5525 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5306 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.963496208190918 time used to do the prediction : 0.19566941261291504 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5304 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5304 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.0948269367218018 time used to do the prediction : 0.17688393592834473 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.8596649169921875e-05 save missing photos in datou_result : time spend for datou_step_exec : 9.900104999542236 time spend to save output : 0.0006711483001708984 total time spend for step 3 : 9.900776147842407 step4:merge_mask_thcl_custom Fri Feb 28 03:14:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 151 chid ids of type : 3663 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1340026538, 1340026522, 1340026513, 1340026473, 1340026456, 1340026452, 1340026110, 1340026109, 1340018842, 1340018836] Looping around the photos to save general results len do output : 10 /1340026538Didn't retrieve data .Didn't retrieve data . /1340026522Didn't retrieve data .Didn't retrieve data . /1340026513Didn't retrieve data .Didn't retrieve data . /1340026473Didn't retrieve data .Didn't retrieve data . /1340026456Didn't retrieve data .Didn't retrieve data . /1340026452Didn't retrieve data .Didn't retrieve data . /1340026110Didn't retrieve data .Didn't retrieve data . /1340026109Didn't retrieve data .Didn't retrieve data . /1340018842Didn't retrieve data .Didn't retrieve data . /1340018836Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026538', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026522', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026513', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026473', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026456', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026452', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026110', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026109', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018842', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018836', None, None, None, None, None, '2621865') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01658463478088379 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.06419086456298828 time spend to save output : 0.0170440673828125 total time spend for step 4 : 0.08123493194580078 step5:rle_unique_nms_with_priority Fri Feb 28 03:14:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 8 nb_hashtags : 2 time to prepare the origin masks : 6.354349136352539 time for calcul the mask position with numpy : 0.27661991119384766 nb_pixel_total : 2929296 time to create 1 rle with new method : 0.7100906372070312 time for calcul the mask position with numpy : 0.03351783752441406 nb_pixel_total : 46624 time to create 1 rle with old method : 0.05276679992675781 time for calcul the mask position with numpy : 0.021616458892822266 nb_pixel_total : 118292 time to create 1 rle with old method : 0.16262006759643555 time for calcul the mask position with numpy : 0.020668506622314453 nb_pixel_total : 3989 time to create 1 rle with old method : 0.005076885223388672 time for calcul the mask position with numpy : 0.03126025199890137 nb_pixel_total : 173483 time to create 1 rle with new method : 0.3559434413909912 time for calcul the mask position with numpy : 0.0439908504486084 nb_pixel_total : 2293 time to create 1 rle with old method : 0.0027942657470703125 time for calcul the mask position with numpy : 0.03825092315673828 nb_pixel_total : 138689 time to create 1 rle with old method : 0.15470194816589355 time for calcul the mask position with numpy : 0.03516435623168945 nb_pixel_total : 301849 time to create 1 rle with new method : 0.7960586547851562 time for calcul the mask position with numpy : 0.1613919734954834 nb_pixel_total : 3335725 time to create 1 rle with new method : 0.3844795227050781 create new chi : 3.390864849090576 time to delete rle : 0.01726222038269043 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 20507 TO DO : save crop sub photo not yet done ! save time : 1.7057013511657715 nb_obj : 19 nb_hashtags : 6 time to prepare the origin masks : 10.954274654388428 time for calcul the mask position with numpy : 0.2568342685699463 nb_pixel_total : 6085714 time to create 1 rle with new method : 0.3632168769836426 time for calcul the mask position with numpy : 0.041281700134277344 nb_pixel_total : 8848 time to create 1 rle with old method : 0.011661529541015625 time for calcul the mask position with numpy : 0.03822684288024902 nb_pixel_total : 16554 time to create 1 rle with old method : 0.01829695701599121 time for calcul the mask position with numpy : 0.0395815372467041 nb_pixel_total : 854 time to create 1 rle with old method : 0.001123189926147461 time for calcul the mask position with numpy : 0.0354914665222168 nb_pixel_total : 16816 time to create 1 rle with old method : 0.019233226776123047 time for calcul the mask position with numpy : 0.034081220626831055 nb_pixel_total : 97718 time to create 1 rle with old method : 0.10563945770263672 time for calcul the mask position with numpy : 0.02621006965637207 nb_pixel_total : 8625 time to create 1 rle with old method : 0.00973057746887207 time for calcul the mask position with numpy : 0.02242732048034668 nb_pixel_total : 104396 time to create 1 rle with old method : 0.1246645450592041 time for calcul the mask position with numpy : 0.02165699005126953 nb_pixel_total : 64949 time to create 1 rle with old method : 0.07522869110107422 time for calcul the mask position with numpy : 0.021981239318847656 nb_pixel_total : 58634 time to create 1 rle with old method : 0.06681394577026367 time for calcul the mask position with numpy : 0.022562265396118164 nb_pixel_total : 371 time to create 1 rle with old method : 0.0004401206970214844 time for calcul the mask position with numpy : 0.021776437759399414 nb_pixel_total : 43596 time to create 1 rle with old method : 0.048431396484375 time for calcul the mask position with numpy : 0.022557497024536133 nb_pixel_total : 84209 time to create 1 rle with old method : 0.09278488159179688 time for calcul the mask position with numpy : 0.02147197723388672 nb_pixel_total : 58812 time to create 1 rle with old method : 0.06681346893310547 time for calcul the mask position with numpy : 0.021625518798828125 nb_pixel_total : 4929 time to create 1 rle with old method : 0.005457162857055664 time for calcul the mask position with numpy : 0.03165102005004883 nb_pixel_total : 103505 time to create 1 rle with old method : 0.11344194412231445 time for calcul the mask position with numpy : 0.04041337966918945 nb_pixel_total : 6720 time to create 1 rle with old method : 0.007284402847290039 time for calcul the mask position with numpy : 0.04082155227661133 nb_pixel_total : 65650 time to create 1 rle with old method : 0.07288551330566406 time for calcul the mask position with numpy : 0.04584836959838867 nb_pixel_total : 100053 time to create 1 rle with old method : 0.11601543426513672 time for calcul the mask position with numpy : 0.0420374870300293 nb_pixel_total : 119287 time to create 1 rle with old method : 0.1381971836090088 create new chi : 2.347320318222046 time to delete rle : 0.0020940303802490234 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 14065 TO DO : save crop sub photo not yet done ! save time : 1.6007471084594727 nb_obj : 4 nb_hashtags : 4 time to prepare the origin masks : 3.538234233856201 time for calcul the mask position with numpy : 0.5389904975891113 nb_pixel_total : 6973759 time to create 1 rle with new method : 0.5936002731323242 time for calcul the mask position with numpy : 0.02497267723083496 nb_pixel_total : 29940 time to create 1 rle with old method : 0.033262014389038086 time for calcul the mask position with numpy : 0.022766828536987305 nb_pixel_total : 27555 time to create 1 rle with old method : 0.03131532669067383 time for calcul the mask position with numpy : 0.0223391056060791 nb_pixel_total : 15519 time to create 1 rle with old method : 0.018765926361083984 time for calcul the mask position with numpy : 0.022868871688842773 nb_pixel_total : 3467 time to create 1 rle with old method : 0.004121541976928711 create new chi : 1.354557752609253 time to delete rle : 0.0004279613494873047 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 3694 TO DO : save crop sub photo not yet done ! save time : 0.29000234603881836 nb_obj : 5 nb_hashtags : 4 time to prepare the origin masks : 3.0754969120025635 time for calcul the mask position with numpy : 0.3412322998046875 nb_pixel_total : 6933504 time to create 1 rle with new method : 0.38864612579345703 time for calcul the mask position with numpy : 0.02452397346496582 nb_pixel_total : 3621 time to create 1 rle with old method : 0.004686594009399414 time for calcul the mask position with numpy : 0.02415180206298828 nb_pixel_total : 21664 time to create 1 rle with old method : 0.024588823318481445 time for calcul the mask position with numpy : 0.023522377014160156 nb_pixel_total : 27523 time to create 1 rle with old method : 0.03386402130126953 time for calcul the mask position with numpy : 0.023847103118896484 nb_pixel_total : 7792 time to create 1 rle with old method : 0.008677005767822266 time for calcul the mask position with numpy : 0.023602962493896484 nb_pixel_total : 56136 time to create 1 rle with old method : 0.06169867515563965 create new chi : 1.0242054462432861 time to delete rle : 0.0004761219024658203 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 4012 TO DO : save crop sub photo not yet done ! save time : 0.46056652069091797 nb_obj : 14 nb_hashtags : 6 time to prepare the origin masks : 6.423585414886475 time for calcul the mask position with numpy : 0.2804715633392334 nb_pixel_total : 6099319 time to create 1 rle with new method : 0.42298316955566406 time for calcul the mask position with numpy : 0.02528238296508789 nb_pixel_total : 240116 time to create 1 rle with new method : 0.43353939056396484 time for calcul the mask position with numpy : 0.02449345588684082 nb_pixel_total : 92 time to create 1 rle with old method : 0.0001971721649169922 time for calcul the mask position with numpy : 0.025032520294189453 nb_pixel_total : 35218 time to create 1 rle with old method : 0.03681302070617676 time for calcul the mask position with numpy : 0.02341318130493164 nb_pixel_total : 1104 time to create 1 rle with old method : 0.0012106895446777344 time for calcul the mask position with numpy : 0.02626800537109375 nb_pixel_total : 83635 time to create 1 rle with old method : 0.08744120597839355 time for calcul the mask position with numpy : 0.024994611740112305 nb_pixel_total : 105342 time to create 1 rle with old method : 0.11269927024841309 time for calcul the mask position with numpy : 0.02320122718811035 nb_pixel_total : 2332 time to create 1 rle with old method : 0.0026197433471679688 time for calcul the mask position with numpy : 0.02337789535522461 nb_pixel_total : 68651 time to create 1 rle with old method : 0.07541990280151367 time for calcul the mask position with numpy : 0.02323436737060547 nb_pixel_total : 6892 time to create 1 rle with old method : 0.007593631744384766 time for calcul the mask position with numpy : 0.02293992042541504 nb_pixel_total : 12528 time to create 1 rle with old method : 0.013218402862548828 time for calcul the mask position with numpy : 0.023631572723388672 nb_pixel_total : 37049 time to create 1 rle with old method : 0.04503583908081055 time for calcul the mask position with numpy : 0.03472328186035156 nb_pixel_total : 49264 time to create 1 rle with old method : 0.05149960517883301 time for calcul the mask position with numpy : 0.039665937423706055 nb_pixel_total : 228182 time to create 1 rle with new method : 0.4807124137878418 time for calcul the mask position with numpy : 0.03877687454223633 nb_pixel_total : 80516 time to create 1 rle with old method : 0.09971284866333008 create new chi : 2.623337984085083 time to delete rle : 0.0010027885437011719 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 10062 TO DO : save crop sub photo not yet done ! save time : 0.9646511077880859 nb_obj : 19 nb_hashtags : 8 time to prepare the origin masks : 10.9450364112854 time for calcul the mask position with numpy : 0.3245832920074463 nb_pixel_total : 5004622 time to create 1 rle with new method : 0.6107251644134521 time for calcul the mask position with numpy : 0.04002022743225098 nb_pixel_total : 50765 time to create 1 rle with old method : 0.0777130126953125 time for calcul the mask position with numpy : 0.03650975227355957 nb_pixel_total : 30945 time to create 1 rle with old method : 0.03435039520263672 time for calcul the mask position with numpy : 0.03516030311584473 nb_pixel_total : 6400 time to create 1 rle with old method : 0.007200002670288086 time for calcul the mask position with numpy : 0.03338003158569336 nb_pixel_total : 627 time to create 1 rle with old method : 0.0006856918334960938 time for calcul the mask position with numpy : 0.038907527923583984 nb_pixel_total : 523292 time to create 1 rle with new method : 0.5529985427856445 time for calcul the mask position with numpy : 0.03328967094421387 nb_pixel_total : 3534 time to create 1 rle with old method : 0.004499673843383789 time for calcul the mask position with numpy : 0.034533023834228516 nb_pixel_total : 3915 time to create 1 rle with old method : 0.004373311996459961 time for calcul the mask position with numpy : 0.0353236198425293 nb_pixel_total : 54341 time to create 1 rle with old method : 0.060384273529052734 time for calcul the mask position with numpy : 0.034108877182006836 nb_pixel_total : 9190 time to create 1 rle with old method : 0.010357141494750977 time for calcul the mask position with numpy : 0.03564167022705078 nb_pixel_total : 31071 time to create 1 rle with old method : 0.0346379280090332 time for calcul the mask position with numpy : 0.03568434715270996 nb_pixel_total : 46128 time to create 1 rle with old method : 0.05548357963562012 time for calcul the mask position with numpy : 0.045239925384521484 nb_pixel_total : 4288 time to create 1 rle with old method : 0.005166292190551758 time for calcul the mask position with numpy : 0.0391077995300293 nb_pixel_total : 397 time to create 1 rle with old method : 0.0008373260498046875 time for calcul the mask position with numpy : 0.0335087776184082 nb_pixel_total : 1435 time to create 1 rle with old method : 0.0020465850830078125 time for calcul the mask position with numpy : 0.03041815757751465 nb_pixel_total : 28530 time to create 1 rle with old method : 0.04428458213806152 time for calcul the mask position with numpy : 0.0440211296081543 nb_pixel_total : 1027369 time to create 1 rle with new method : 0.7743768692016602 time for calcul the mask position with numpy : 0.025282859802246094 nb_pixel_total : 123417 time to create 1 rle with old method : 0.1432037353515625 time for calcul the mask position with numpy : 0.02278923988342285 nb_pixel_total : 2266 time to create 1 rle with old method : 0.002550363540649414 time for calcul the mask position with numpy : 0.022049427032470703 nb_pixel_total : 97708 time to create 1 rle with old method : 0.11572694778442383 create new chi : 3.6093661785125732 time to delete rle : 0.0014643669128417969 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 15611 TO DO : save crop sub photo not yet done ! save time : 1.3299617767333984 nb_obj : 22 nb_hashtags : 7 time to prepare the origin masks : 15.679019927978516 time for calcul the mask position with numpy : 0.49744296073913574 nb_pixel_total : 6027749 time to create 1 rle with new method : 0.7822885513305664 time for calcul the mask position with numpy : 0.04618096351623535 nb_pixel_total : 5155 time to create 1 rle with old method : 0.006381034851074219 time for calcul the mask position with numpy : 0.04214024543762207 nb_pixel_total : 108451 time to create 1 rle with old method : 0.1156461238861084 time for calcul the mask position with numpy : 0.03987288475036621 nb_pixel_total : 17952 time to create 1 rle with old method : 0.024486541748046875 time for calcul the mask position with numpy : 0.039225101470947266 nb_pixel_total : 9589 time to create 1 rle with old method : 0.010628938674926758 time for calcul the mask position with numpy : 0.038407087326049805 nb_pixel_total : 23188 time to create 1 rle with old method : 0.025180816650390625 time for calcul the mask position with numpy : 0.04016709327697754 nb_pixel_total : 70338 time to create 1 rle with old method : 0.07822227478027344 time for calcul the mask position with numpy : 0.04107856750488281 nb_pixel_total : 49416 time to create 1 rle with old method : 0.05634427070617676 time for calcul the mask position with numpy : 0.039124250411987305 nb_pixel_total : 10607 time to create 1 rle with old method : 0.01203775405883789 time for calcul the mask position with numpy : 0.04403829574584961 nb_pixel_total : 293317 time to create 1 rle with new method : 1.009718894958496 time for calcul the mask position with numpy : 0.037134408950805664 nb_pixel_total : 87916 time to create 1 rle with old method : 0.090301513671875 time for calcul the mask position with numpy : 0.03741717338562012 nb_pixel_total : 3524 time to create 1 rle with old method : 0.0037496089935302734 time for calcul the mask position with numpy : 0.036344051361083984 nb_pixel_total : 2104 time to create 1 rle with old method : 0.002291440963745117 time for calcul the mask position with numpy : 0.03943061828613281 nb_pixel_total : 15167 time to create 1 rle with old method : 0.0160369873046875 time for calcul the mask position with numpy : 0.041380882263183594 nb_pixel_total : 1407 time to create 1 rle with old method : 0.0015261173248291016 time for calcul the mask position with numpy : 0.03986048698425293 nb_pixel_total : 55225 time to create 1 rle with old method : 0.059578657150268555 time for calcul the mask position with numpy : 0.039807796478271484 nb_pixel_total : 1902 time to create 1 rle with old method : 0.002092599868774414 time for calcul the mask position with numpy : 0.04241061210632324 nb_pixel_total : 4660 time to create 1 rle with old method : 0.005120754241943359 time for calcul the mask position with numpy : 0.038773536682128906 nb_pixel_total : 13496 time to create 1 rle with old method : 0.014880180358886719 time for calcul the mask position with numpy : 0.041314125061035156 nb_pixel_total : 29770 time to create 1 rle with old method : 0.03219294548034668 time for calcul the mask position with numpy : 0.04667234420776367 nb_pixel_total : 16989 time to create 1 rle with old method : 0.018106698989868164 time for calcul the mask position with numpy : 0.04084324836730957 nb_pixel_total : 115745 time to create 1 rle with old method : 0.1205604076385498 time for calcul the mask position with numpy : 0.04232072830200195 nb_pixel_total : 86573 time to create 1 rle with old method : 0.09318280220031738 create new chi : 4.041908025741577 time to delete rle : 0.0023729801177978516 batch 1 Loaded 23 chid ids of type : 3726 Number RLEs to save : 16216 TO DO : save crop sub photo not yet done ! save time : 1.228623867034912 nb_obj : 19 nb_hashtags : 6 time to prepare the origin masks : 13.947664976119995 time for calcul the mask position with numpy : 0.3535134792327881 nb_pixel_total : 5013241 time to create 1 rle with new method : 0.8369026184082031 time for calcul the mask position with numpy : 0.031348228454589844 nb_pixel_total : 18852 time to create 1 rle with old method : 0.023785114288330078 time for calcul the mask position with numpy : 0.03435158729553223 nb_pixel_total : 8058 time to create 1 rle with old method : 0.009089231491088867 time for calcul the mask position with numpy : 0.03485369682312012 nb_pixel_total : 77272 time to create 1 rle with old method : 0.09425926208496094 time for calcul the mask position with numpy : 0.03437185287475586 nb_pixel_total : 4067 time to create 1 rle with old method : 0.004862785339355469 time for calcul the mask position with numpy : 0.03758573532104492 nb_pixel_total : 219418 time to create 1 rle with new method : 0.5119898319244385 time for calcul the mask position with numpy : 0.03438091278076172 nb_pixel_total : 22849 time to create 1 rle with old method : 0.02681732177734375 time for calcul the mask position with numpy : 0.03495073318481445 nb_pixel_total : 101214 time to create 1 rle with old method : 0.11392998695373535 time for calcul the mask position with numpy : 0.03329205513000488 nb_pixel_total : 2072 time to create 1 rle with old method : 0.002368450164794922 time for calcul the mask position with numpy : 0.03387260437011719 nb_pixel_total : 8777 time to create 1 rle with old method : 0.010042428970336914 time for calcul the mask position with numpy : 0.03116583824157715 nb_pixel_total : 9672 time to create 1 rle with old method : 0.010817766189575195 time for calcul the mask position with numpy : 0.0323944091796875 nb_pixel_total : 937953 time to create 1 rle with new method : 0.6228406429290771 time for calcul the mask position with numpy : 0.021427631378173828 nb_pixel_total : 18092 time to create 1 rle with old method : 0.02016592025756836 time for calcul the mask position with numpy : 0.022585630416870117 nb_pixel_total : 145077 time to create 1 rle with old method : 0.16180419921875 time for calcul the mask position with numpy : 0.021123647689819336 nb_pixel_total : 21598 time to create 1 rle with old method : 0.02432417869567871 time for calcul the mask position with numpy : 0.020984888076782227 nb_pixel_total : 4131 time to create 1 rle with old method : 0.0046956539154052734 time for calcul the mask position with numpy : 0.021959543228149414 nb_pixel_total : 30678 time to create 1 rle with old method : 0.03533577919006348 time for calcul the mask position with numpy : 0.023004770278930664 nb_pixel_total : 274174 time to create 1 rle with new method : 0.3904407024383545 time for calcul the mask position with numpy : 0.023118019104003906 nb_pixel_total : 96197 time to create 1 rle with old method : 0.10845494270324707 time for calcul the mask position with numpy : 0.02252364158630371 nb_pixel_total : 36848 time to create 1 rle with old method : 0.04209589958190918 create new chi : 4.060405254364014 time to delete rle : 0.0017805099487304688 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 17219 TO DO : save crop sub photo not yet done ! save time : 1.2320725917816162 nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 11.989377975463867 time for calcul the mask position with numpy : 0.3157973289489746 nb_pixel_total : 5519490 time to create 1 rle with new method : 0.6643264293670654 time for calcul the mask position with numpy : 0.04149651527404785 nb_pixel_total : 13835 time to create 1 rle with old method : 0.01637101173400879 time for calcul the mask position with numpy : 0.029529809951782227 nb_pixel_total : 29416 time to create 1 rle with old method : 0.08020997047424316 time for calcul the mask position with numpy : 0.09095215797424316 nb_pixel_total : 5816 time to create 1 rle with old method : 0.010260581970214844 time for calcul the mask position with numpy : 0.08820652961730957 nb_pixel_total : 18405 time to create 1 rle with old method : 0.03860974311828613 time for calcul the mask position with numpy : 0.033370256423950195 nb_pixel_total : 5975 time to create 1 rle with old method : 0.010813236236572266 time for calcul the mask position with numpy : 0.033042192459106445 nb_pixel_total : 4055 time to create 1 rle with old method : 0.007526874542236328 time for calcul the mask position with numpy : 0.03294563293457031 nb_pixel_total : 211347 time to create 1 rle with new method : 0.8853814601898193 time for calcul the mask position with numpy : 0.0356287956237793 nb_pixel_total : 902 time to create 1 rle with old method : 0.0011196136474609375 time for calcul the mask position with numpy : 0.035355567932128906 nb_pixel_total : 36957 time to create 1 rle with old method : 0.048578739166259766 time for calcul the mask position with numpy : 0.03441762924194336 nb_pixel_total : 71583 time to create 1 rle with old method : 0.08454608917236328 time for calcul the mask position with numpy : 0.04829883575439453 nb_pixel_total : 929605 time to create 1 rle with new method : 0.5186805725097656 time for calcul the mask position with numpy : 0.03730368614196777 nb_pixel_total : 119564 time to create 1 rle with old method : 0.13133931159973145 time for calcul the mask position with numpy : 0.0341646671295166 nb_pixel_total : 8305 time to create 1 rle with old method : 0.009272575378417969 time for calcul the mask position with numpy : 0.02588176727294922 nb_pixel_total : 53833 time to create 1 rle with old method : 0.05900073051452637 time for calcul the mask position with numpy : 0.022857189178466797 nb_pixel_total : 4281 time to create 1 rle with old method : 0.006918907165527344 time for calcul the mask position with numpy : 0.023368358612060547 nb_pixel_total : 5725 time to create 1 rle with old method : 0.00919795036315918 time for calcul the mask position with numpy : 0.02362203598022461 nb_pixel_total : 11146 time to create 1 rle with old method : 0.01483917236328125 create new chi : 3.678640127182007 time to delete rle : 0.0014925003051757812 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 13977 TO DO : save crop sub photo not yet done ! save time : 2.266584873199463 nb_obj : 17 nb_hashtags : 5 time to prepare the origin masks : 11.321109294891357 time for calcul the mask position with numpy : 0.3594822883605957 nb_pixel_total : 6241753 time to create 1 rle with new method : 0.4686439037322998 time for calcul the mask position with numpy : 0.03441166877746582 nb_pixel_total : 29057 time to create 1 rle with old method : 0.0339505672454834 time for calcul the mask position with numpy : 0.0339665412902832 nb_pixel_total : 32441 time to create 1 rle with old method : 0.052706241607666016 time for calcul the mask position with numpy : 0.02628350257873535 nb_pixel_total : 32461 time to create 1 rle with old method : 0.040894508361816406 time for calcul the mask position with numpy : 0.022923946380615234 nb_pixel_total : 25793 time to create 1 rle with old method : 0.029434680938720703 time for calcul the mask position with numpy : 0.02271294593811035 nb_pixel_total : 54509 time to create 1 rle with old method : 0.0614931583404541 time for calcul the mask position with numpy : 0.022945880889892578 nb_pixel_total : 1104 time to create 1 rle with old method : 0.0014576911926269531 time for calcul the mask position with numpy : 0.025590181350708008 nb_pixel_total : 126951 time to create 1 rle with old method : 0.1549367904663086 time for calcul the mask position with numpy : 0.027067899703979492 nb_pixel_total : 42088 time to create 1 rle with old method : 0.048483848571777344 time for calcul the mask position with numpy : 0.02498030662536621 nb_pixel_total : 117956 time to create 1 rle with old method : 0.13913798332214355 time for calcul the mask position with numpy : 0.024876832962036133 nb_pixel_total : 2875 time to create 1 rle with old method : 0.004091024398803711 time for calcul the mask position with numpy : 0.026276588439941406 nb_pixel_total : 26031 time to create 1 rle with old method : 0.031190872192382812 time for calcul the mask position with numpy : 0.024011611938476562 nb_pixel_total : 4431 time to create 1 rle with old method : 0.005583047866821289 time for calcul the mask position with numpy : 0.02400970458984375 nb_pixel_total : 82086 time to create 1 rle with old method : 0.10250163078308105 time for calcul the mask position with numpy : 0.023550033569335938 nb_pixel_total : 10192 time to create 1 rle with old method : 0.01165151596069336 time for calcul the mask position with numpy : 0.024273157119750977 nb_pixel_total : 105566 time to create 1 rle with old method : 0.11324286460876465 time for calcul the mask position with numpy : 0.020319223403930664 nb_pixel_total : 66314 time to create 1 rle with old method : 0.06881237030029297 time for calcul the mask position with numpy : 0.0203855037689209 nb_pixel_total : 48632 time to create 1 rle with old method : 0.05104374885559082 create new chi : 2.2472984790802 time to delete rle : 0.0010557174682617188 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 11213 TO DO : save crop sub photo not yet done ! save time : 0.7915694713592529 map_output_result : {1340026538: (0.0, 'Should be the crop_list due to order', 0), 1340026522: (0.0, 'Should be the crop_list due to order', 0), 1340026513: (0.0, 'Should be the crop_list due to order', 0), 1340026473: (0.0, 'Should be the crop_list due to order', 0), 1340026456: (0.0, 'Should be the crop_list due to order', 0), 1340026452: (0.0, 'Should be the crop_list due to order', 0), 1340026110: (0.0, 'Should be the crop_list due to order', 0), 1340026109: (0.0, 'Should be the crop_list due to order', 0), 1340018842: (0.0, 'Should be the crop_list due to order', 0), 1340018836: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1340026538, 1340026522, 1340026513, 1340026473, 1340026456, 1340026452, 1340026110, 1340026109, 1340018842, 1340018836] Looping around the photos to save general results len do output : 10 /1340026538.Didn't retrieve data . /1340026522.Didn't retrieve data . /1340026513.Didn't retrieve data . /1340026473.Didn't retrieve data . /1340026456.Didn't retrieve data . /1340026452.Didn't retrieve data . /1340026110.Didn't retrieve data . /1340026109.Didn't retrieve data . /1340018842.Didn't retrieve data . /1340018836.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026538', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026522', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026513', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026473', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026456', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026452', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026110', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026109', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018842', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018836', None, None, None, None, None, '2621865') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.019478321075439453 save_final save missing photos in datou_result : time spend for datou_step_exec : 135.9082911014557 time spend to save output : 0.01992487907409668 total time spend for step 5 : 135.92821598052979 step6:crop_condition Fri Feb 28 03:16:45 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 160 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740709007_2684112 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.0393288135528564 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740709012_2684112 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.863231658935547 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740709018_2684112 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.2347533702850342 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740709021_2684112 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.75980806350708 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! About to upload 7 photos upload in portfolio : 4869462 init cache_photo without model_param we have 7 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740709025_2684112 we have uploaded 7 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.140678644180298 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740709030_2684112 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.9328508377075195 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! About to upload 1 photos upload in portfolio : 4869462 init cache_photo without model_param we have 1 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740709032_2684112 we have uploaded 1 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.921389102935791 we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1340026538, 1340026522, 1340026513, 1340026473, 1340026456, 1340026452, 1340026110, 1340026109, 1340018842, 1340018836] Looping around the photos to save general results len do output : 29 /1340048265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048356Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048358Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048360Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048362Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048364Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048366Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048368Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048370Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048372Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048374Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048412Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048413Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048414Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048415Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048453Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048454Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048455Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048456Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048457Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048458Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048459Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048460Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048461Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048469Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048471Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1340048509Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026538', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026522', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026513', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026473', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026456', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026452', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026110', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026109', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018842', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018836', None, None, None, None, None, '2621865') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 97 time used for this insertion : 0.03793215751647949 save_final save missing photos in datou_result : time spend for datou_step_exec : 32.65638709068298 time spend to save output : 0.03891706466674805 total time spend for step 6 : 32.69530415534973 step7:ventilate_hashtags_in_portfolio Fri Feb 28 03:17:17 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20905796 get user id for portfolio 20905796 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20905796 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','Teint_Dans_La_Masse','papier','Carton_brun','cartonnette','plastique','kraft','metal','environnement','flou','mal_croppe','autre_refus')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20905796 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','Teint_Dans_La_Masse','papier','Carton_brun','cartonnette','plastique','kraft','metal','environnement','flou','mal_croppe','autre_refus')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20905796 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton_gris','Teint_Dans_La_Masse','papier','Carton_brun','cartonnette','plastique','kraft','metal','environnement','flou','mal_croppe','autre_refus')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20906043,20906044,20906045,20906046,20906047,20906048,20906049,20906050,20906051,20906052,20906053,20906054?tags=Carton_gris,Teint_Dans_La_Masse,papier,Carton_brun,cartonnette,plastique,kraft,metal,environnement,flou,mal_croppe,autre_refus Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1340026538, 1340026522, 1340026513, 1340026473, 1340026456, 1340026452, 1340026110, 1340026109, 1340018842, 1340018836] Looping around the photos to save general results len do output : 1 /20905796. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026538', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026522', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026513', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026473', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026456', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026452', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026110', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026109', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018842', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018836', None, None, None, None, None, '2621865') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.01964545249938965 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.087615966796875 time spend to save output : 0.01990532875061035 total time spend for step 7 : 1.1075212955474854 step8:final Fri Feb 28 03:17:18 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1340026538: ('0.0662392077948395',), 1340026522: ('0.0662392077948395',), 1340026513: ('0.0662392077948395',), 1340026473: ('0.0662392077948395',), 1340026456: ('0.0662392077948395',), 1340026452: ('0.0662392077948395',), 1340026110: ('0.0662392077948395',), 1340026109: ('0.0662392077948395',), 1340018842: ('0.0662392077948395',), 1340018836: ('0.0662392077948395',)} new output for save of step final : {1340026538: ('0.0662392077948395',), 1340026522: ('0.0662392077948395',), 1340026513: ('0.0662392077948395',), 1340026473: ('0.0662392077948395',), 1340026456: ('0.0662392077948395',), 1340026452: ('0.0662392077948395',), 1340026110: ('0.0662392077948395',), 1340026109: ('0.0662392077948395',), 1340018842: ('0.0662392077948395',), 1340018836: ('0.0662392077948395',)} [1340026538, 1340026522, 1340026513, 1340026473, 1340026456, 1340026452, 1340026110, 1340026109, 1340018842, 1340018836] Looping around the photos to save general results len do output : 10 /1340026538.Didn't retrieve data . /1340026522.Didn't retrieve data . /1340026513.Didn't retrieve data . /1340026473.Didn't retrieve data . /1340026456.Didn't retrieve data . /1340026452.Didn't retrieve data . /1340026110.Didn't retrieve data . /1340026109.Didn't retrieve data . /1340018842.Didn't retrieve data . /1340018836.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026538', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026522', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026513', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026473', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026456', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026452', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026110', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026109', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018842', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018836', None, None, None, None, None, '2621865') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01404571533203125 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11456060409545898 time spend to save output : 0.014471292495727539 total time spend for step 8 : 0.12903189659118652 step9:velours_tree Fri Feb 28 03:17:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.04792165756225586 time spend to save output : 0.00010251998901367188 total time spend for step 9 : 0.04802417755126953 step10:send_mail_cod Fri Feb 28 03:17:19 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20905796_28-02-2025_03_17_19.pdf 20906047 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette209060471740709039 20906048 change filename to text .change filename to text .change filename to text .change filename to text .imagette209060481740709039 20906049 change filename to text .change filename to text .change filename to text .imagette209060491740709039 20906050 imagette209060501740709039 20906052 imagette209060521740709039 20906053 imagette209060531740709039 20906054 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette209060541740709039 20906043 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette209060431740709040 20906044 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette209060441740709041 20906046 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette209060461740709042 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20905796 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20906043,20906044,20906045,20906046,20906047,20906048,20906049,20906050,20906051,20906052,20906053,20906054?tags=Carton_gris,Teint_Dans_La_Masse,papier,Carton_brun,cartonnette,plastique,kraft,metal,environnement,flou,mal_croppe,autre_refus your option no_mail is active, we will not send the real mail to your client args[1340026538] : ('0.0662392077948395',) no score found for photo 1340026538 We are sending mail with results at report@fotonower.com args[1340026522] : ('0.0662392077948395',) no score found for photo 1340026522 We are sending mail with results at report@fotonower.com args[1340026513] : ('0.0662392077948395',) no score found for photo 1340026513 We are sending mail with results at report@fotonower.com args[1340026473] : ('0.0662392077948395',) no score found for photo 1340026473 We are sending mail with results at report@fotonower.com args[1340026456] : ('0.0662392077948395',) no score found for photo 1340026456 We are sending mail with results at report@fotonower.com args[1340026452] : ('0.0662392077948395',) no score found for photo 1340026452 We are sending mail with results at report@fotonower.com args[1340026110] : ('0.0662392077948395',) no score found for photo 1340026110 We are sending mail with results at report@fotonower.com args[1340026109] : ('0.0662392077948395',) no score found for photo 1340026109 We are sending mail with results at report@fotonower.com args[1340018842] : ('0.0662392077948395',) no score found for photo 1340018842 We are sending mail with results at report@fotonower.com args[1340018836] : ('0.0662392077948395',) no score found for photo 1340018836 We are sending mail with results at report@fotonower.com refus_total : 0.0662392077948395 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20905796 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1340018826,1340026473,1340026456,1340026452,1340026110,1340026109,1340018842,1339928322,1340018829,1340026513,1340026522,1339929112,1339928413,1339928409,1339928405,1340018836,1340026538,1340018786,1340018755,1339928400) Found this number of photos: 20 begin to download photo : 1340018826 begin to download photo : 1340026109 begin to download photo : 1340026522 begin to download photo : 1340018836 download finish for photo 1340026109 begin to download photo : 1340018842 download finish for photo 1340018826 begin to download photo : 1340026473 download finish for photo 1340026522 begin to download photo : 1339929112 download finish for photo 1340018836 begin to download photo : 1340026538 download finish for photo 1340026473 begin to download photo : 1340026456 download finish for photo 1340018842 begin to download photo : 1339928322 download finish for photo 1340026538 begin to download photo : 1340018786 download finish for photo 1339929112 begin to download photo : 1339928413 download finish for photo 1340026456 begin to download photo : 1340026452 download finish for photo 1339928322 begin to download photo : 1340018829 download finish for photo 1339928413 begin to download photo : 1339928409 download finish for photo 1340026452 begin to download photo : 1340026110 download finish for photo 1340018786 begin to download photo : 1340018755 download finish for photo 1340018829 begin to download photo : 1340026513 download finish for photo 1340018755 begin to download photo : 1339928400 download finish for photo 1340026110 download finish for photo 1340026513 download finish for photo 1339928409 begin to download photo : 1339928405 download finish for photo 1339928400 download finish for photo 1339928405 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20905796_28-02-2025_03_17_19.pdf results_Auto_P20905796_28-02-2025_03_17_19.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20905796_28-02-2025_03_17_19.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20905796','results_Auto_P20905796_28-02-2025_03_17_19.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20905796_28-02-2025_03_17_19.pdf','pdf','','0.93','0.0662392077948395') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1340026538, 1340026522, 1340026513, 1340026473, 1340026456, 1340026452, 1340026110, 1340026109, 1340018842, 1340018836] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026538', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026522', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026513', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026473', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026456', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026452', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026110', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026109', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018842', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018836', None, None, None, None, None, '2621865') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.0168607234954834 save_final save missing photos in datou_result : time spend for datou_step_exec : 8.742794513702393 time spend to save output : 0.01718282699584961 total time spend for step 10 : 8.759977340698242 step11:split_time_score Fri Feb 28 03:17:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('15', 12), ('13', 8)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 27022025 20905796 Nombre de photos uploadées : 20 / 23040 (0%) 27022025 20905796 Nombre de photos taguées (types de déchets): 0 / 20 (0%) 27022025 20905796 Nombre de photos taguées (volume) : 0 / 20 (0%) elapsed_time : load_data_split_time_score 2.384185791015625e-06 elapsed_time : order_list_meta_photo_and_scores 5.4836273193359375e-06 ???????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0005688667297363281 elapsed_time : insert_dashboard_record_day_entry 0.03270411491394043 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901267 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901268 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901269 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901270 order by id desc limit 1 Qualite : 0.07406922604400437 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901271_28-02-2025_02_07_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901271 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901271 AND mptpi.`type`=4207 To do Qualite : 0.08224904095108558 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901272_28-02-2025_01_50_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901272 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901272 AND mptpi.`type`=4207 To do Qualite : 0.13080982673511443 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901273_28-02-2025_02_26_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901273 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901273 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20905767 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20902989 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901760 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901761 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20905768 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20905769 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20905782 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20905783 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20902990 order by id desc limit 1 Qualite : 0.0662392077948395 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20905796_28-02-2025_03_17_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20905796 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20905796 AND mptpi.`type`=3726 To do Qualite : 0.07982605740529508 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901763_28-02-2025_02_56_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901763 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11524 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11533 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11532 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11528 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11528 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11531 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11531 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11525 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11578 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11578 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11527 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11526 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11526 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11535 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11527 doesn't seem to be define in the database( WARNING : type of input 3 of step 11526 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11524 doesn't seem to be define in the database( WARNING : type of input 2 of step 11528 doesn't seem to be define in the database( WARNING : output 1 of step 11524 have datatype=2 whereas input 1 of step 11531 have datatype=7 WARNING : type of output 2 of step 11531 doesn't seem to be define in the database( WARNING : type of input 1 of step 11525 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11527 have datatype=10 whereas input 3 of step 11534 have datatype=6 WARNING : type of input 2 of step 11578 doesn't seem to be define in the database( WARNING : output 1 of step 11525 have datatype=7 whereas input 2 of step 11578 have datatype=None WARNING : type of output 3 of step 11578 doesn't seem to be define in the database( WARNING : type of input 1 of step 11527 doesn't seem to be define in the database( WARNING : output 0 of step 11527 have datatype=10 whereas input 0 of step 11584 have datatype=18 WARNING : type of input 5 of step 11534 doesn't seem to be define in the database( WARNING : output 0 of step 11584 have datatype=11 whereas input 5 of step 11534 have datatype=None WARNING : type of output 1 of step 11533 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : type of output 1 of step 11532 doesn't seem to be define in the database( WARNING : type of input 3 of step 11528 doesn't seem to be define in the database( WARNING : output 0 of step 11531 have datatype=1 whereas input 0 of step 11525 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901763 AND mptpi.`type`=4211 To do Qualite : 0.1992153555709031 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901764_28-02-2025_02_59_22.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901764 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901764 AND mptpi.`type`=3327 To do Qualite : 0.19853583693421006 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901765_28-02-2025_02_40_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901765 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901765 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901276 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901766 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901278 order by id desc limit 1 Qualite : 0.191975621870083 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901279_28-02-2025_03_04_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901279 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901279 AND mptpi.`type`=3594 To do find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901767_28-02-2025_01_59_06.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901767 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901768 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901769_28-02-2025_01_39_56.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901769 order by id desc limit 1 Qualite : 0.08650521466694104 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20902992_28-02-2025_03_01_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20902992 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20902992 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20905797 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901283 order by id desc limit 1 Qualite : 0.1199733468175425 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901284_28-02-2025_01_54_57.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901284 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901284 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901285 order by id desc limit 1 Qualite : 0.16412012537548684 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901286_28-02-2025_02_31_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901286 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901286 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901287 order by id desc limit 1 Qualite : 0.21264894399468545 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901288_28-02-2025_02_10_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901288 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901288 AND mptpi.`type`=3594 To do Qualite : 0.0662862777318847 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901301_28-02-2025_00_57_08.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901301 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901301 AND mptpi.`type`=4207 To do Qualite : 0.10326228661809093 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901314_28-02-2025_01_38_23.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901314 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901314 AND mptpi.`type`=4207 To do Qualite : 0.059403225739113 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20901315_28-02-2025_01_08_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20901315 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20901315 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'27022025': {'nb_upload': 20, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1340026538, 1340026522, 1340026513, 1340026473, 1340026456, 1340026452, 1340026110, 1340026109, 1340018842, 1340018836] Looping around the photos to save general results len do output : 1 /20905796Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026538', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026522', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026513', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026473', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026456', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026452', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026110', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340026109', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018842', None, None, None, None, None, '2621865') ('3459', None, None, None, None, None, None, None, '2621865') ('3459', '20905796', '1340018836', None, None, None, None, None, '2621865') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.014217138290405273 save_final save missing photos in datou_result : time spend for datou_step_exec : 19.705363750457764 time spend to save output : 0.014462709426879883 total time spend for step 11 : 19.719826459884644 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 144.91user 128.63system 5:23.19elapsed 84%CPU (0avgtext+0avgdata 4320764maxresident)k 725032inputs+126696outputs (1067major+13161229minor)pagefaults 0swaps