python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2772460 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 27>10 please execute split_portfolio.py -i 20785558 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2611126'] with mtr_portfolio_ids : ['20785558'] and first list_photo_ids : [] new path : /proc/2772460/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 2.06864070892334 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Tue Feb 25 00:42:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3183 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-25 00:42:33.372716: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-25 00:42:33.407071: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-25 00:42:33.409548: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe694000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-25 00:42:33.409600: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-25 00:42:33.414143: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-25 00:42:33.666228: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xcdf6b70 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-25 00:42:33.666285: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-25 00:42:33.667554: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-25 00:42:33.668209: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-25 00:42:33.672794: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-25 00:42:33.692989: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-25 00:42:33.694195: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-25 00:42:33.729491: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-25 00:42:33.731153: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-25 00:42:33.792794: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-25 00:42:33.793870: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-25 00:42:33.794458: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-25 00:42:33.795005: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-25 00:42:33.795021: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-25 00:42:33.795029: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-25 00:42:33.796237: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2731 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-25 00:42:34.090192: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-25 00:42:34.090289: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-25 00:42:34.090311: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-25 00:42:34.090329: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-25 00:42:34.090348: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-25 00:42:34.090366: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-25 00:42:34.090383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-25 00:42:34.090401: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-25 00:42:34.091468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-25 00:42:34.092553: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-25 00:42:34.092598: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-25 00:42:34.092617: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-25 00:42:34.092634: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-25 00:42:34.092651: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-25 00:42:34.092667: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-25 00:42:34.092684: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-25 00:42:34.092701: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-25 00:42:34.093707: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-25 00:42:34.093738: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-25 00:42:34.093749: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-25 00:42:34.093758: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-25 00:42:34.094814: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2731 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-25 00:42:45.707159: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-25 00:42:45.876403: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-25 00:42:47.504490: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.504544: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.505296: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.505317: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.512502: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.512520: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.513053: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.513067: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.519532: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.519552: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.520136: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.520152: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.550610: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.550633: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.551202: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.551229: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.556970: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.556989: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.557523: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.557537: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-25 00:42:47.591051: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.591653: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.593472: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.594012: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.632045: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.632604: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.634490: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.635046: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.659543: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.660146: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.661601: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.662138: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.672044: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.672642: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.674251: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.674787: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.680616: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.681343: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.683099: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.683829: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.710356: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.710922: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.711499: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.712087: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.715958: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.716539: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.732732: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.733278: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.733813: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.734346: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.746954: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.747534: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.748123: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.748687: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.752840: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.753382: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.757887: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.758428: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.770429: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.770992: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.775116: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.775708: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.776295: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.776847: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.777518: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.778061: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.788746: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.789287: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.789838: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.790375: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.790950: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.791542: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.806119: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.806660: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.853944: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.854522: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.862270: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.862801: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.885403: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.886482: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.887589: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.888178: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.892114: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.892649: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.893178: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.893706: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.895176: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.895193: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-25 00:42:47.904677: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.905210: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.913952: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.914484: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.915026: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.915553: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.916086: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-25 00:42:47.916611: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 20 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 4 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 21 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 6 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 27 Detection mask done ! Trying to reset tf kernel 2772797 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1990 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3183 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.0008356571197509766 nb_pixel_total : 41325 time to create 1 rle with old method : 0.045879364013671875 length of segment : 274 time for calcul the mask position with numpy : 0.0004258155822753906 nb_pixel_total : 20427 time to create 1 rle with old method : 0.021566152572631836 length of segment : 254 time for calcul the mask position with numpy : 0.00035071372985839844 nb_pixel_total : 23945 time to create 1 rle with old method : 0.02506232261657715 length of segment : 214 time for calcul the mask position with numpy : 0.031210899353027344 nb_pixel_total : 1426276 time to create 1 rle with new method : 1.107970952987671 length of segment : 2051 time for calcul the mask position with numpy : 0.0019235610961914062 nb_pixel_total : 105799 time to create 1 rle with old method : 0.11764121055603027 length of segment : 366 time for calcul the mask position with numpy : 0.0034072399139404297 nb_pixel_total : 176394 time to create 1 rle with new method : 0.011641740798950195 length of segment : 715 time for calcul the mask position with numpy : 0.001501321792602539 nb_pixel_total : 116877 time to create 1 rle with old method : 0.12405920028686523 length of segment : 320 time for calcul the mask position with numpy : 0.00026416778564453125 nb_pixel_total : 13735 time to create 1 rle with old method : 0.015636444091796875 length of segment : 142 time for calcul the mask position with numpy : 0.00010180473327636719 nb_pixel_total : 3710 time to create 1 rle with old method : 0.004247188568115234 length of segment : 47 time for calcul the mask position with numpy : 0.0065348148345947266 nb_pixel_total : 352611 time to create 1 rle with new method : 0.02429342269897461 length of segment : 1552 time for calcul the mask position with numpy : 0.004831075668334961 nb_pixel_total : 127402 time to create 1 rle with old method : 0.1323854923248291 length of segment : 305 time for calcul the mask position with numpy : 0.009433984756469727 nb_pixel_total : 625395 time to create 1 rle with new method : 0.033759355545043945 length of segment : 1235 time for calcul the mask position with numpy : 0.0027256011962890625 nb_pixel_total : 169285 time to create 1 rle with new method : 0.007648468017578125 length of segment : 549 time for calcul the mask position with numpy : 0.003775358200073242 nb_pixel_total : 133653 time to create 1 rle with old method : 0.13946771621704102 length of segment : 787 time for calcul the mask position with numpy : 0.0028753280639648438 nb_pixel_total : 148416 time to create 1 rle with old method : 0.15938377380371094 length of segment : 481 time for calcul the mask position with numpy : 0.0017936229705810547 nb_pixel_total : 98928 time to create 1 rle with old method : 0.10653567314147949 length of segment : 524 time for calcul the mask position with numpy : 0.0012865066528320312 nb_pixel_total : 78509 time to create 1 rle with old method : 0.08018326759338379 length of segment : 527 time for calcul the mask position with numpy : 0.003989696502685547 nb_pixel_total : 197757 time to create 1 rle with new method : 0.013196229934692383 length of segment : 1147 time for calcul the mask position with numpy : 0.0005128383636474609 nb_pixel_total : 12081 time to create 1 rle with old method : 0.013221263885498047 length of segment : 128 time for calcul the mask position with numpy : 0.0007975101470947266 nb_pixel_total : 22379 time to create 1 rle with old method : 0.023823261260986328 length of segment : 202 time for calcul the mask position with numpy : 0.0006654262542724609 nb_pixel_total : 16717 time to create 1 rle with old method : 0.017597198486328125 length of segment : 217 time for calcul the mask position with numpy : 0.0026497840881347656 nb_pixel_total : 97489 time to create 1 rle with old method : 0.10570240020751953 length of segment : 305 time for calcul the mask position with numpy : 0.003985404968261719 nb_pixel_total : 183287 time to create 1 rle with new method : 0.0062961578369140625 length of segment : 551 time for calcul the mask position with numpy : 0.0007588863372802734 nb_pixel_total : 35322 time to create 1 rle with old method : 0.03885030746459961 length of segment : 116 time for calcul the mask position with numpy : 0.00013589859008789062 nb_pixel_total : 1740 time to create 1 rle with old method : 0.0020864009857177734 length of segment : 60 time for calcul the mask position with numpy : 0.0006036758422851562 nb_pixel_total : 20329 time to create 1 rle with old method : 0.024188995361328125 length of segment : 189 time for calcul the mask position with numpy : 0.0013415813446044922 nb_pixel_total : 43796 time to create 1 rle with old method : 0.04807400703430176 length of segment : 273 time for calcul the mask position with numpy : 0.0024306774139404297 nb_pixel_total : 89420 time to create 1 rle with old method : 0.10721015930175781 length of segment : 368 time for calcul the mask position with numpy : 0.000278472900390625 nb_pixel_total : 2117 time to create 1 rle with old method : 0.0027608871459960938 length of segment : 83 time for calcul the mask position with numpy : 0.002755403518676758 nb_pixel_total : 191590 time to create 1 rle with new method : 0.006752490997314453 length of segment : 579 time for calcul the mask position with numpy : 0.00015091896057128906 nb_pixel_total : 1345 time to create 1 rle with old method : 0.001764535903930664 length of segment : 47 time for calcul the mask position with numpy : 0.0035266876220703125 nb_pixel_total : 102394 time to create 1 rle with old method : 0.11595487594604492 length of segment : 546 time for calcul the mask position with numpy : 0.0006203651428222656 nb_pixel_total : 16671 time to create 1 rle with old method : 0.021659135818481445 length of segment : 126 time for calcul the mask position with numpy : 0.0016036033630371094 nb_pixel_total : 49012 time to create 1 rle with old method : 0.05244088172912598 length of segment : 359 time for calcul the mask position with numpy : 0.007384300231933594 nb_pixel_total : 223054 time to create 1 rle with new method : 0.013327598571777344 length of segment : 748 time for calcul the mask position with numpy : 0.0013103485107421875 nb_pixel_total : 36032 time to create 1 rle with old method : 0.04131317138671875 length of segment : 252 time for calcul the mask position with numpy : 0.0007984638214111328 nb_pixel_total : 30115 time to create 1 rle with old method : 0.03549814224243164 length of segment : 278 time for calcul the mask position with numpy : 0.0007064342498779297 nb_pixel_total : 27488 time to create 1 rle with old method : 0.029717683792114258 length of segment : 216 time for calcul the mask position with numpy : 0.0007011890411376953 nb_pixel_total : 36599 time to create 1 rle with old method : 0.04086136817932129 length of segment : 250 time for calcul the mask position with numpy : 0.0004937648773193359 nb_pixel_total : 27531 time to create 1 rle with old method : 0.032645225524902344 length of segment : 221 time for calcul the mask position with numpy : 0.0002143383026123047 nb_pixel_total : 2655 time to create 1 rle with old method : 0.0030126571655273438 length of segment : 88 time for calcul the mask position with numpy : 0.0002872943878173828 nb_pixel_total : 5119 time to create 1 rle with old method : 0.006165981292724609 length of segment : 104 time for calcul the mask position with numpy : 0.0006172657012939453 nb_pixel_total : 12849 time to create 1 rle with old method : 0.01458287239074707 length of segment : 129 time for calcul the mask position with numpy : 0.004304170608520508 nb_pixel_total : 236287 time to create 1 rle with new method : 0.012082576751708984 length of segment : 1131 time for calcul the mask position with numpy : 0.001178741455078125 nb_pixel_total : 69373 time to create 1 rle with old method : 0.07519197463989258 length of segment : 364 time for calcul the mask position with numpy : 0.00015807151794433594 nb_pixel_total : 1614 time to create 1 rle with old method : 0.0020792484283447266 length of segment : 51 time for calcul the mask position with numpy : 0.0009593963623046875 nb_pixel_total : 63110 time to create 1 rle with old method : 0.06926727294921875 length of segment : 488 time for calcul the mask position with numpy : 0.0029730796813964844 nb_pixel_total : 80269 time to create 1 rle with old method : 0.0865623950958252 length of segment : 968 time for calcul the mask position with numpy : 0.0003986358642578125 nb_pixel_total : 4491 time to create 1 rle with old method : 0.004998683929443359 length of segment : 164 time for calcul the mask position with numpy : 0.7556538581848145 nb_pixel_total : 4019099 time to create 1 rle with new method : 0.7810931205749512 length of segment : 3196 time for calcul the mask position with numpy : 0.0013430118560791016 nb_pixel_total : 74526 time to create 1 rle with old method : 0.08066534996032715 length of segment : 520 time for calcul the mask position with numpy : 0.0013670921325683594 nb_pixel_total : 69934 time to create 1 rle with old method : 0.07512736320495605 length of segment : 504 time for calcul the mask position with numpy : 0.0008029937744140625 nb_pixel_total : 35418 time to create 1 rle with old method : 0.04136347770690918 length of segment : 230 time for calcul the mask position with numpy : 0.26453185081481934 nb_pixel_total : 2621962 time to create 1 rle with new method : 0.5572082996368408 length of segment : 2917 time for calcul the mask position with numpy : 0.0022335052490234375 nb_pixel_total : 82275 time to create 1 rle with old method : 0.09603071212768555 length of segment : 611 time for calcul the mask position with numpy : 0.0005941390991210938 nb_pixel_total : 23481 time to create 1 rle with old method : 0.02595353126525879 length of segment : 221 time for calcul the mask position with numpy : 0.014113664627075195 nb_pixel_total : 569947 time to create 1 rle with new method : 0.24959111213684082 length of segment : 1321 time for calcul the mask position with numpy : 0.0005908012390136719 nb_pixel_total : 48370 time to create 1 rle with old method : 0.055765628814697266 length of segment : 278 time for calcul the mask position with numpy : 0.0005509853363037109 nb_pixel_total : 13119 time to create 1 rle with old method : 0.014765024185180664 length of segment : 140 time for calcul the mask position with numpy : 0.001039743423461914 nb_pixel_total : 46944 time to create 1 rle with old method : 0.05653572082519531 length of segment : 407 time for calcul the mask position with numpy : 0.001995086669921875 nb_pixel_total : 66644 time to create 1 rle with old method : 0.07678008079528809 length of segment : 381 time for calcul the mask position with numpy : 0.0010428428649902344 nb_pixel_total : 39895 time to create 1 rle with old method : 0.043295860290527344 length of segment : 276 time for calcul the mask position with numpy : 0.005391120910644531 nb_pixel_total : 167378 time to create 1 rle with new method : 0.009162664413452148 length of segment : 518 time for calcul the mask position with numpy : 0.0014777183532714844 nb_pixel_total : 34338 time to create 1 rle with old method : 0.04046988487243652 length of segment : 420 time for calcul the mask position with numpy : 0.0005826950073242188 nb_pixel_total : 10108 time to create 1 rle with old method : 0.01157689094543457 length of segment : 202 time for calcul the mask position with numpy : 0.0027704238891601562 nb_pixel_total : 41288 time to create 1 rle with old method : 0.04906654357910156 length of segment : 473 time for calcul the mask position with numpy : 0.009874582290649414 nb_pixel_total : 451048 time to create 1 rle with new method : 0.015242576599121094 length of segment : 959 time for calcul the mask position with numpy : 0.0008294582366943359 nb_pixel_total : 23825 time to create 1 rle with old method : 0.028113126754760742 length of segment : 288 time for calcul the mask position with numpy : 0.0009577274322509766 nb_pixel_total : 39878 time to create 1 rle with old method : 0.04446101188659668 length of segment : 220 time for calcul the mask position with numpy : 0.0002999305725097656 nb_pixel_total : 8576 time to create 1 rle with old method : 0.009140968322753906 length of segment : 111 time for calcul the mask position with numpy : 0.0004069805145263672 nb_pixel_total : 6921 time to create 1 rle with old method : 0.008051633834838867 length of segment : 133 time for calcul the mask position with numpy : 0.00019931793212890625 nb_pixel_total : 1437 time to create 1 rle with old method : 0.0019164085388183594 length of segment : 86 time for calcul the mask position with numpy : 0.004120349884033203 nb_pixel_total : 165508 time to create 1 rle with new method : 0.006781101226806641 length of segment : 534 time for calcul the mask position with numpy : 0.0007910728454589844 nb_pixel_total : 35549 time to create 1 rle with old method : 0.03766059875488281 length of segment : 290 time for calcul the mask position with numpy : 0.0030040740966796875 nb_pixel_total : 120484 time to create 1 rle with old method : 0.1332237720489502 length of segment : 535 time for calcul the mask position with numpy : 0.002185344696044922 nb_pixel_total : 109303 time to create 1 rle with old method : 0.12043261528015137 length of segment : 454 time for calcul the mask position with numpy : 0.0006034374237060547 nb_pixel_total : 9618 time to create 1 rle with old method : 0.010957002639770508 length of segment : 177 time for calcul the mask position with numpy : 0.0001895427703857422 nb_pixel_total : 4901 time to create 1 rle with old method : 0.0054171085357666016 length of segment : 57 time for calcul the mask position with numpy : 0.0004963874816894531 nb_pixel_total : 23480 time to create 1 rle with old method : 0.024778127670288086 length of segment : 203 time for calcul the mask position with numpy : 0.0006120204925537109 nb_pixel_total : 14275 time to create 1 rle with old method : 0.01601386070251465 length of segment : 222 time for calcul the mask position with numpy : 0.001953601837158203 nb_pixel_total : 78332 time to create 1 rle with old method : 0.08148050308227539 length of segment : 397 time for calcul the mask position with numpy : 0.0003147125244140625 nb_pixel_total : 7796 time to create 1 rle with old method : 0.008892297744750977 length of segment : 171 time for calcul the mask position with numpy : 0.0003750324249267578 nb_pixel_total : 10405 time to create 1 rle with old method : 0.012189865112304688 length of segment : 134 time for calcul the mask position with numpy : 0.0006973743438720703 nb_pixel_total : 21009 time to create 1 rle with old method : 0.02317523956298828 length of segment : 228 time for calcul the mask position with numpy : 0.004262447357177734 nb_pixel_total : 215963 time to create 1 rle with new method : 0.006425380706787109 length of segment : 432 time for calcul the mask position with numpy : 0.0024313926696777344 nb_pixel_total : 106181 time to create 1 rle with old method : 0.12111830711364746 length of segment : 536 time for calcul the mask position with numpy : 0.0021266937255859375 nb_pixel_total : 125962 time to create 1 rle with old method : 0.13859963417053223 length of segment : 864 time for calcul the mask position with numpy : 0.00023293495178222656 nb_pixel_total : 2926 time to create 1 rle with old method : 0.0038721561431884766 length of segment : 61 time for calcul the mask position with numpy : 0.00048542022705078125 nb_pixel_total : 17239 time to create 1 rle with old method : 0.019947290420532227 length of segment : 183 time for calcul the mask position with numpy : 0.0004010200500488281 nb_pixel_total : 8028 time to create 1 rle with old method : 0.009562015533447266 length of segment : 170 time for calcul the mask position with numpy : 0.0021572113037109375 nb_pixel_total : 89534 time to create 1 rle with old method : 0.10239815711975098 length of segment : 478 time for calcul the mask position with numpy : 0.004144906997680664 nb_pixel_total : 177457 time to create 1 rle with new method : 0.0071849822998046875 length of segment : 743 time for calcul the mask position with numpy : 0.0007181167602539062 nb_pixel_total : 15261 time to create 1 rle with old method : 0.017010927200317383 length of segment : 247 time for calcul the mask position with numpy : 0.0008914470672607422 nb_pixel_total : 30407 time to create 1 rle with old method : 0.03240346908569336 length of segment : 253 time for calcul the mask position with numpy : 0.0028765201568603516 nb_pixel_total : 178490 time to create 1 rle with new method : 0.006856441497802734 length of segment : 651 time for calcul the mask position with numpy : 0.0001678466796875 nb_pixel_total : 5361 time to create 1 rle with old method : 0.006296634674072266 length of segment : 136 time for calcul the mask position with numpy : 0.3232264518737793 nb_pixel_total : 3851080 time to create 1 rle with new method : 0.653266191482544 length of segment : 3163 time for calcul the mask position with numpy : 0.0004096031188964844 nb_pixel_total : 24226 time to create 1 rle with old method : 0.02687692642211914 length of segment : 266 time for calcul the mask position with numpy : 0.0003769397735595703 nb_pixel_total : 23255 time to create 1 rle with old method : 0.024707317352294922 length of segment : 282 time for calcul the mask position with numpy : 0.0037262439727783203 nb_pixel_total : 280158 time to create 1 rle with new method : 0.009962081909179688 length of segment : 773 time for calcul the mask position with numpy : 0.0007014274597167969 nb_pixel_total : 46261 time to create 1 rle with old method : 0.05395913124084473 length of segment : 212 time for calcul the mask position with numpy : 0.00018835067749023438 nb_pixel_total : 8389 time to create 1 rle with old method : 0.010050535202026367 length of segment : 117 time for calcul the mask position with numpy : 0.013822078704833984 nb_pixel_total : 740099 time to create 1 rle with new method : 0.048140764236450195 length of segment : 1610 time for calcul the mask position with numpy : 0.0519254207611084 nb_pixel_total : 2638386 time to create 1 rle with new method : 0.5085248947143555 length of segment : 3719 time for calcul the mask position with numpy : 0.0008153915405273438 nb_pixel_total : 48590 time to create 1 rle with old method : 0.0685420036315918 length of segment : 261 time for calcul the mask position with numpy : 0.0013649463653564453 nb_pixel_total : 105436 time to create 1 rle with old method : 0.10652017593383789 length of segment : 374 time for calcul the mask position with numpy : 0.00018644332885742188 nb_pixel_total : 8792 time to create 1 rle with old method : 0.010050296783447266 length of segment : 108 time for calcul the mask position with numpy : 0.00011134147644042969 nb_pixel_total : 2703 time to create 1 rle with old method : 0.002864837646484375 length of segment : 117 time for calcul the mask position with numpy : 0.001039743423461914 nb_pixel_total : 95933 time to create 1 rle with old method : 0.10511088371276855 length of segment : 422 time for calcul the mask position with numpy : 0.0025713443756103516 nb_pixel_total : 69783 time to create 1 rle with old method : 0.07321572303771973 length of segment : 366 time for calcul the mask position with numpy : 0.0023555755615234375 nb_pixel_total : 93560 time to create 1 rle with old method : 0.10184049606323242 length of segment : 375 time for calcul the mask position with numpy : 0.0012831687927246094 nb_pixel_total : 37638 time to create 1 rle with old method : 0.04095888137817383 length of segment : 208 time for calcul the mask position with numpy : 0.0003139972686767578 nb_pixel_total : 4450 time to create 1 rle with old method : 0.005015850067138672 length of segment : 122 time for calcul the mask position with numpy : 0.0012359619140625 nb_pixel_total : 46374 time to create 1 rle with old method : 0.04819965362548828 length of segment : 257 time for calcul the mask position with numpy : 0.00045561790466308594 nb_pixel_total : 9613 time to create 1 rle with old method : 0.0106658935546875 length of segment : 131 time for calcul the mask position with numpy : 0.0015435218811035156 nb_pixel_total : 44551 time to create 1 rle with old method : 0.047638893127441406 length of segment : 229 time for calcul the mask position with numpy : 0.0001971721649169922 nb_pixel_total : 2378 time to create 1 rle with old method : 0.0028536319732666016 length of segment : 83 time for calcul the mask position with numpy : 0.0024442672729492188 nb_pixel_total : 56468 time to create 1 rle with old method : 0.0586092472076416 length of segment : 495 time for calcul the mask position with numpy : 0.0007846355438232422 nb_pixel_total : 17833 time to create 1 rle with old method : 0.020703554153442383 length of segment : 171 time for calcul the mask position with numpy : 0.0009465217590332031 nb_pixel_total : 21114 time to create 1 rle with old method : 0.02451944351196289 length of segment : 286 time for calcul the mask position with numpy : 0.002049684524536133 nb_pixel_total : 42523 time to create 1 rle with old method : 0.0489041805267334 length of segment : 322 time for calcul the mask position with numpy : 0.0003876686096191406 nb_pixel_total : 13043 time to create 1 rle with old method : 0.015439748764038086 length of segment : 232 time for calcul the mask position with numpy : 0.00030541419982910156 nb_pixel_total : 6300 time to create 1 rle with old method : 0.0071794986724853516 length of segment : 84 time for calcul the mask position with numpy : 0.001371145248413086 nb_pixel_total : 40436 time to create 1 rle with old method : 0.04444766044616699 length of segment : 294 time for calcul the mask position with numpy : 0.0012798309326171875 nb_pixel_total : 47057 time to create 1 rle with old method : 0.054268598556518555 length of segment : 288 time for calcul the mask position with numpy : 0.004747152328491211 nb_pixel_total : 132137 time to create 1 rle with old method : 0.14702677726745605 length of segment : 796 time for calcul the mask position with numpy : 0.0057065486907958984 nb_pixel_total : 227032 time to create 1 rle with new method : 0.00814962387084961 length of segment : 539 time for calcul the mask position with numpy : 0.0004925727844238281 nb_pixel_total : 21079 time to create 1 rle with old method : 0.025159120559692383 length of segment : 145 time for calcul the mask position with numpy : 0.00019860267639160156 nb_pixel_total : 3464 time to create 1 rle with old method : 0.003983020782470703 length of segment : 125 time for calcul the mask position with numpy : 0.000946044921875 nb_pixel_total : 28778 time to create 1 rle with old method : 0.03420281410217285 length of segment : 167 time for calcul the mask position with numpy : 0.0005021095275878906 nb_pixel_total : 4709 time to create 1 rle with old method : 0.0062007904052734375 length of segment : 109 time for calcul the mask position with numpy : 0.0006570816040039062 nb_pixel_total : 11767 time to create 1 rle with old method : 0.013138771057128906 length of segment : 143 time spent for convertir_results : 21.928364992141724 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 132 chid ids of type : 3663 Number RLEs to save : 61050 save missing photos in datou_result : time spend for datou_step_exec : 70.39502477645874 time spend to save output : 4.19295334815979 total time spend for step 1 : 74.58797812461853 step2:crop_condition Tue Feb 25 00:43:44 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 132 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 9 About to insert : list_path_to_insert length 9 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 39 About to insert : list_path_to_insert length 39 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 1 About to insert : list_path_to_insert length 1 new photo from crops ! we have finished the crop for the class : kraft begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339415827, 1339415737, 1339415732, 1339415700, 1339415655, 1339415650, 1339415646, 1339322976, 1339322973, 1339322946] Looping around the photos to save general results len do output : 77 /-3685828805Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828827Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828841Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828836Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828865Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828885Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828903Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828911Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828914Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828851Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828862Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828879Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828873Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828896Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828891Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828909Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828913Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828920Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828929Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828815Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828828Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828893Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828820Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828819Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828810Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828811Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828818Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828809Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828826Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828832Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828834Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828831Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828821Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828825Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828822Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828847Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828843Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828849Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828848Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828852Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828837Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828856Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828859Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828860Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828863Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828878Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828877Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828866Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828870Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828897Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828884Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828899Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828898Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828895Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828904Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828930Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828925Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828918Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828916Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828931Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828924Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828806Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828823Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828824Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828840Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828842Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828855Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828872Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828888Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828915Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828919Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828923Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828883Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828890Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828905Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828910Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3685828844Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415827', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415737', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415732', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415700', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415655', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415650', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415646', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322976', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322973', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322946', None, None, None, None, None, '2611126') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 241 time used for this insertion : 0.03968477249145508 save_final save missing photos in datou_result : time spend for datou_step_exec : 36.386515855789185 time spend to save output : 0.04146528244018555 total time spend for step 2 : 36.42798113822937 step3:thcl Tue Feb 25 00:44:20 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.008725166320800781 time to convert the images to numpy array : 1.8258945941925049 time to import caffe and check if the image exist : 0.013414621353149414 time to convert the images to numpy array : 1.8279139995574951 time to import caffe and check if the image exist : 0.013330459594726562 time to convert the images to numpy array : 1.850522756576538 time to import caffe and check if the image exist : 0.01178741455078125 time to convert the images to numpy array : 1.8604602813720703 time to import caffe and check if the image exist : 0.010821342468261719 time to convert the images to numpy array : 1.8623108863830566 time to import caffe and check if the image exist : 0.010118961334228516 time to convert the images to numpy array : 1.877269983291626 time to import caffe and check if the image exist : 0.00874948501586914 time to convert the images to numpy array : 2.4498202800750732 time to import caffe and check if the image exist : 0.014397144317626953 time to convert the images to numpy array : 2.4540016651153564 time to import caffe and check if the image exist : 0.011512279510498047 time to convert the images to numpy array : 2.4589028358459473 time to import caffe and check if the image exist : 0.013086318969726562 time to convert the images to numpy array : 2.471424102783203 total time to convert the images to numpy array : 2.756469964981079 list photo_ids error: [] list photo_ids correct : [-3685828883, -3685828890, -3685828905, -3685828910, -3685828844, -3685828840, -3685828842, -3685828855, -3685828872, -3685828888, -3685828915, -3685828919, -3685828923, -3685828805, -3685828827, -3685828841, -3685828836, -3685828865, -3685828885, -3685828903, -3685828911, -3685828925, -3685828918, -3685828916, -3685828931, -3685828924, -3685828806, -3685828823, -3685828824, -3685828837, -3685828856, -3685828859, -3685828860, -3685828863, -3685828878, -3685828877, -3685828866, -3685828913, -3685828920, -3685828929, -3685828815, -3685828828, -3685828893, -3685828820, -3685828819, -3685828914, -3685828851, -3685828862, -3685828879, -3685828873, -3685828896, -3685828891, -3685828909, -3685828870, -3685828897, -3685828884, -3685828899, -3685828898, -3685828895, -3685828904, -3685828930, -3685828810, -3685828811, -3685828818, -3685828809, -3685828826, -3685828832, -3685828834, -3685828831, -3685828821, -3685828825, -3685828822, -3685828847, -3685828843, -3685828849, -3685828848, -3685828852] number of photos to traite : 77 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3183 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2964 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.9060451984405518 time used to do the prediction : 0.20432448387145996 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2962 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2962 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.8759558200836182 time used to do the prediction : 0.15497636795043945 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.71661376953125e-05 save missing photos in datou_result : time spend for datou_step_exec : 11.932640552520752 time spend to save output : 0.0005621910095214844 total time spend for step 3 : 11.933202743530273 step4:merge_mask_thcl_custom Tue Feb 25 00:44:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 132 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1339415827, 1339415737, 1339415732, 1339415700, 1339415655, 1339415650, 1339415646, 1339322976, 1339322973, 1339322946] Looping around the photos to save general results len do output : 10 /1339415827Didn't retrieve data .Didn't retrieve data . /1339415737Didn't retrieve data .Didn't retrieve data . /1339415732Didn't retrieve data .Didn't retrieve data . /1339415700Didn't retrieve data .Didn't retrieve data . /1339415655Didn't retrieve data .Didn't retrieve data . /1339415650Didn't retrieve data .Didn't retrieve data . /1339415646Didn't retrieve data .Didn't retrieve data . /1339322976Didn't retrieve data .Didn't retrieve data . /1339322973Didn't retrieve data .Didn't retrieve data . /1339322946Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415827', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415737', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415732', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415700', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415655', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415650', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415646', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322976', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322973', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322946', None, None, None, None, None, '2611126') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014270782470703125 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.0772249698638916 time spend to save output : 0.014779329299926758 total time spend for step 4 : 0.09200429916381836 step5:rle_unique_nms_with_priority Tue Feb 25 00:44:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 16 nb_hashtags : 6 time to prepare the origin masks : 12.433714628219604 time for calcul the mask position with numpy : 0.3545413017272949 nb_pixel_total : 4386399 time to create 1 rle with new method : 0.812546968460083 time for calcul the mask position with numpy : 0.020127534866333008 nb_pixel_total : 13064 time to create 1 rle with old method : 0.014007568359375 time for calcul the mask position with numpy : 0.01943063735961914 nb_pixel_total : 4577 time to create 1 rle with old method : 0.0051212310791015625 time for calcul the mask position with numpy : 0.020115137100219727 nb_pixel_total : 127178 time to create 1 rle with old method : 0.12879085540771484 time for calcul the mask position with numpy : 0.02076435089111328 nb_pixel_total : 11074 time to create 1 rle with old method : 0.012387990951538086 time for calcul the mask position with numpy : 0.02120375633239746 nb_pixel_total : 175825 time to create 1 rle with new method : 0.5469317436218262 time for calcul the mask position with numpy : 0.02184748649597168 nb_pixel_total : 169214 time to create 1 rle with new method : 0.6935968399047852 time for calcul the mask position with numpy : 0.02094125747680664 nb_pixel_total : 99485 time to create 1 rle with old method : 0.10686731338500977 time for calcul the mask position with numpy : 0.02236795425415039 nb_pixel_total : 113192 time to create 1 rle with old method : 0.11868500709533691 time for calcul the mask position with numpy : 0.02250051498413086 nb_pixel_total : 239413 time to create 1 rle with new method : 0.5489058494567871 time for calcul the mask position with numpy : 0.02033829689025879 nb_pixel_total : 3697 time to create 1 rle with old method : 0.003988504409790039 time for calcul the mask position with numpy : 0.01963186264038086 nb_pixel_total : 41259 time to create 1 rle with old method : 0.04279899597167969 time for calcul the mask position with numpy : 0.019946813583374023 nb_pixel_total : 13638 time to create 1 rle with old method : 0.014158487319946289 time for calcul the mask position with numpy : 0.19924187660217285 nb_pixel_total : 1424055 time to create 1 rle with new method : 0.5535886287689209 time for calcul the mask position with numpy : 0.020938873291015625 nb_pixel_total : 98635 time to create 1 rle with old method : 0.10096573829650879 time for calcul the mask position with numpy : 0.020595788955688477 nb_pixel_total : 23892 time to create 1 rle with old method : 0.02451777458190918 time for calcul the mask position with numpy : 0.020326614379882812 nb_pixel_total : 105643 time to create 1 rle with old method : 0.10855317115783691 create new chi : 4.814838886260986 time to delete rle : 0.016929149627685547 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 18938 TO DO : save crop sub photo not yet done ! save time : 1.0232007503509521 nb_obj : 15 nb_hashtags : 5 time to prepare the origin masks : 11.720264673233032 time for calcul the mask position with numpy : 0.5885837078094482 nb_pixel_total : 6241065 time to create 1 rle with new method : 0.6156570911407471 time for calcul the mask position with numpy : 0.03256869316101074 nb_pixel_total : 12068 time to create 1 rle with old method : 0.012380838394165039 time for calcul the mask position with numpy : 0.020746946334838867 nb_pixel_total : 96952 time to create 1 rle with old method : 0.09982109069824219 time for calcul the mask position with numpy : 0.020901203155517578 nb_pixel_total : 78598 time to create 1 rle with old method : 0.08015871047973633 time for calcul the mask position with numpy : 0.019632577896118164 nb_pixel_total : 1335 time to create 1 rle with old method : 0.0014195442199707031 time for calcul the mask position with numpy : 0.01969170570373535 nb_pixel_total : 2102 time to create 1 rle with old method : 0.002168893814086914 time for calcul the mask position with numpy : 0.01992487907409668 nb_pixel_total : 20212 time to create 1 rle with old method : 0.02081584930419922 time for calcul the mask position with numpy : 0.019486427307128906 nb_pixel_total : 1726 time to create 1 rle with old method : 0.0018362998962402344 time for calcul the mask position with numpy : 0.019200801849365234 nb_pixel_total : 9563 time to create 1 rle with old method : 0.010818243026733398 time for calcul the mask position with numpy : 0.019565582275390625 nb_pixel_total : 22307 time to create 1 rle with old method : 0.023726224899291992 time for calcul the mask position with numpy : 0.0198209285736084 nb_pixel_total : 35184 time to create 1 rle with old method : 0.037796974182128906 time for calcul the mask position with numpy : 0.021854877471923828 nb_pixel_total : 197036 time to create 1 rle with new method : 0.8153157234191895 time for calcul the mask position with numpy : 0.020702600479125977 nb_pixel_total : 16658 time to create 1 rle with old method : 0.017902612686157227 time for calcul the mask position with numpy : 0.020739078521728516 nb_pixel_total : 43576 time to create 1 rle with old method : 0.04491543769836426 time for calcul the mask position with numpy : 0.021982431411743164 nb_pixel_total : 182852 time to create 1 rle with new method : 0.709191083908081 time for calcul the mask position with numpy : 0.020951271057128906 nb_pixel_total : 89006 time to create 1 rle with old method : 0.09190559387207031 create new chi : 3.5677833557128906 time to delete rle : 0.0009551048278808594 batch 1 Loaded 16 chid ids of type : 3726 Number RLEs to save : 11266 TO DO : save crop sub photo not yet done ! save time : 2.2567319869995117 nb_obj : 18 nb_hashtags : 6 time to prepare the origin masks : 12.63667893409729 time for calcul the mask position with numpy : 0.694739580154419 nb_pixel_total : 6219316 time to create 1 rle with new method : 0.634645938873291 time for calcul the mask position with numpy : 0.019352436065673828 nb_pixel_total : 480 time to create 1 rle with old method : 0.0006816387176513672 time for calcul the mask position with numpy : 0.019376039505004883 nb_pixel_total : 3434 time to create 1 rle with old method : 0.0037059783935546875 time for calcul the mask position with numpy : 0.020524978637695312 nb_pixel_total : 27429 time to create 1 rle with old method : 0.028500080108642578 time for calcul the mask position with numpy : 0.019644975662231445 nb_pixel_total : 4487 time to create 1 rle with old method : 0.004601478576660156 time for calcul the mask position with numpy : 0.02367854118347168 nb_pixel_total : 59610 time to create 1 rle with old method : 0.06102252006530762 time for calcul the mask position with numpy : 0.02116990089416504 nb_pixel_total : 1602 time to create 1 rle with old method : 0.0018548965454101562 time for calcul the mask position with numpy : 0.0313875675201416 nb_pixel_total : 12781 time to create 1 rle with old method : 0.014292240142822266 time for calcul the mask position with numpy : 0.03372645378112793 nb_pixel_total : 16575 time to create 1 rle with old method : 0.018106698989868164 time for calcul the mask position with numpy : 0.03410601615905762 nb_pixel_total : 760 time to create 1 rle with old method : 0.0012617111206054688 time for calcul the mask position with numpy : 0.033379316329956055 nb_pixel_total : 36430 time to create 1 rle with old method : 0.038617849349975586 time for calcul the mask position with numpy : 0.021310806274414062 nb_pixel_total : 5069 time to create 1 rle with old method : 0.0056416988372802734 time for calcul the mask position with numpy : 0.021115541458129883 nb_pixel_total : 2640 time to create 1 rle with old method : 0.0029840469360351562 time for calcul the mask position with numpy : 0.02163529396057129 nb_pixel_total : 94158 time to create 1 rle with old method : 0.10188174247741699 time for calcul the mask position with numpy : 0.025727510452270508 nb_pixel_total : 179248 time to create 1 rle with new method : 0.6440892219543457 time for calcul the mask position with numpy : 0.022494077682495117 nb_pixel_total : 227256 time to create 1 rle with new method : 1.0395393371582031 time for calcul the mask position with numpy : 0.0269472599029541 nb_pixel_total : 30075 time to create 1 rle with old method : 0.042670488357543945 time for calcul the mask position with numpy : 0.024768590927124023 nb_pixel_total : 48911 time to create 1 rle with old method : 0.05442023277282715 time for calcul the mask position with numpy : 0.025693655014038086 nb_pixel_total : 79979 time to create 1 rle with old method : 0.08954167366027832 create new chi : 4.015068769454956 time to delete rle : 0.0012574195861816406 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 12746 TO DO : save crop sub photo not yet done ! save time : 2.7459566593170166 nb_obj : 3 nb_hashtags : 3 time to prepare the origin masks : 3.522230625152588 time for calcul the mask position with numpy : 0.2610645294189453 nb_pixel_total : 2930611 time to create 1 rle with new method : 0.7020833492279053 time for calcul the mask position with numpy : 0.0195925235748291 nb_pixel_total : 6901 time to create 1 rle with old method : 0.0076334476470947266 time for calcul the mask position with numpy : 0.020122528076171875 nb_pixel_total : 68635 time to create 1 rle with old method : 0.07120871543884277 time for calcul the mask position with numpy : 0.3668491840362549 nb_pixel_total : 4044093 time to create 1 rle with new method : 0.6869661808013916 create new chi : 2.1947197914123535 time to delete rle : 0.0008325576782226562 batch 1 Loaded 4 chid ids of type : 3726 Number RLEs to save : 9696 TO DO : save crop sub photo not yet done ! save time : 0.8958466053009033 nb_obj : 8 nb_hashtags : 4 time to prepare the origin masks : 7.957294702529907 time for calcul the mask position with numpy : 0.13767313957214355 nb_pixel_total : 3685218 time to create 1 rle with new method : 1.2162768840789795 time for calcul the mask position with numpy : 0.021819591522216797 nb_pixel_total : 46715 time to create 1 rle with old method : 0.05094480514526367 time for calcul the mask position with numpy : 0.02190542221069336 nb_pixel_total : 13081 time to create 1 rle with old method : 0.014363765716552734 time for calcul the mask position with numpy : 0.027089595794677734 nb_pixel_total : 483143 time to create 1 rle with new method : 0.33785152435302734 time for calcul the mask position with numpy : 0.02570962905883789 nb_pixel_total : 48616 time to create 1 rle with old method : 0.052782297134399414 time for calcul the mask position with numpy : 0.11261200904846191 nb_pixel_total : 2632282 time to create 1 rle with new method : 0.9404704570770264 time for calcul the mask position with numpy : 0.040650129318237305 nb_pixel_total : 35397 time to create 1 rle with old method : 0.04996538162231445 time for calcul the mask position with numpy : 0.037123680114746094 nb_pixel_total : 82382 time to create 1 rle with old method : 0.09314179420471191 time for calcul the mask position with numpy : 0.0364224910736084 nb_pixel_total : 23406 time to create 1 rle with old method : 0.02589559555053711 create new chi : 3.3264710903167725 time to delete rle : 0.0018169879913330078 batch 1 Loaded 9 chid ids of type : 3726 Number RLEs to save : 13858 TO DO : save crop sub photo not yet done ! save time : 1.2723817825317383 nb_obj : 18 nb_hashtags : 6 time to prepare the origin masks : 16.01017713546753 time for calcul the mask position with numpy : 0.6869759559631348 nb_pixel_total : 5806507 time to create 1 rle with new method : 0.96791672706604 time for calcul the mask position with numpy : 0.033982038497924805 nb_pixel_total : 119993 time to create 1 rle with old method : 0.12455916404724121 time for calcul the mask position with numpy : 0.0368037223815918 nb_pixel_total : 39784 time to create 1 rle with old method : 0.04220104217529297 time for calcul the mask position with numpy : 0.03348374366760254 nb_pixel_total : 66595 time to create 1 rle with old method : 0.07120394706726074 time for calcul the mask position with numpy : 0.03392910957336426 nb_pixel_total : 4882 time to create 1 rle with old method : 0.005360126495361328 time for calcul the mask position with numpy : 0.033217430114746094 nb_pixel_total : 9566 time to create 1 rle with old method : 0.010398387908935547 time for calcul the mask position with numpy : 0.03255629539489746 nb_pixel_total : 1423 time to create 1 rle with old method : 0.0015978813171386719 time for calcul the mask position with numpy : 0.03245258331298828 nb_pixel_total : 6893 time to create 1 rle with old method : 0.007402181625366211 time for calcul the mask position with numpy : 0.02036905288696289 nb_pixel_total : 23451 time to create 1 rle with old method : 0.02482891082763672 time for calcul the mask position with numpy : 0.020940780639648438 nb_pixel_total : 8235 time to create 1 rle with old method : 0.008509397506713867 time for calcul the mask position with numpy : 0.02400827407836914 nb_pixel_total : 39789 time to create 1 rle with old method : 0.04044532775878906 time for calcul the mask position with numpy : 0.02460193634033203 nb_pixel_total : 447565 time to create 1 rle with new method : 0.7354345321655273 time for calcul the mask position with numpy : 0.03339099884033203 nb_pixel_total : 35396 time to create 1 rle with old method : 0.03769278526306152 time for calcul the mask position with numpy : 0.03201460838317871 nb_pixel_total : 9087 time to create 1 rle with old method : 0.009174346923828125 time for calcul the mask position with numpy : 0.03179812431335449 nb_pixel_total : 34280 time to create 1 rle with old method : 0.03482317924499512 time for calcul the mask position with numpy : 0.03154754638671875 nb_pixel_total : 23841 time to create 1 rle with old method : 0.024472951889038086 time for calcul the mask position with numpy : 0.03253936767578125 nb_pixel_total : 166192 time to create 1 rle with new method : 0.5428054332733154 time for calcul the mask position with numpy : 0.03218984603881836 nb_pixel_total : 165477 time to create 1 rle with new method : 0.9170937538146973 time for calcul the mask position with numpy : 0.027463912963867188 nb_pixel_total : 41284 time to create 1 rle with old method : 0.043940067291259766 create new chi : 4.985776901245117 time to delete rle : 0.0012824535369873047 batch 1 Loaded 19 chid ids of type : 3726 Number RLEs to save : 13657 TO DO : save crop sub photo not yet done ! save time : 1.460705041885376 nb_obj : 17 nb_hashtags : 4 time to prepare the origin masks : 12.754150629043579 time for calcul the mask position with numpy : 0.6570353507995605 nb_pixel_total : 6213010 time to create 1 rle with new method : 0.9847161769866943 time for calcul the mask position with numpy : 0.03212738037109375 nb_pixel_total : 5218 time to create 1 rle with old method : 0.005378007888793945 time for calcul the mask position with numpy : 0.03174400329589844 nb_pixel_total : 2216 time to create 1 rle with old method : 0.002508401870727539 time for calcul the mask position with numpy : 0.03157186508178711 nb_pixel_total : 14204 time to create 1 rle with old method : 0.014760017395019531 time for calcul the mask position with numpy : 0.03244209289550781 nb_pixel_total : 8003 time to create 1 rle with old method : 0.02489304542541504 time for calcul the mask position with numpy : 0.03968334197998047 nb_pixel_total : 78252 time to create 1 rle with old method : 0.08625245094299316 time for calcul the mask position with numpy : 0.032652854919433594 nb_pixel_total : 2912 time to create 1 rle with old method : 0.0031414031982421875 time for calcul the mask position with numpy : 0.032627105712890625 nb_pixel_total : 11660 time to create 1 rle with old method : 0.012028217315673828 time for calcul the mask position with numpy : 0.03184008598327637 nb_pixel_total : 15234 time to create 1 rle with old method : 0.015870094299316406 time for calcul the mask position with numpy : 0.021233320236206055 nb_pixel_total : 126044 time to create 1 rle with old method : 0.12946796417236328 time for calcul the mask position with numpy : 0.02180624008178711 nb_pixel_total : 215115 time to create 1 rle with new method : 0.7289361953735352 time for calcul the mask position with numpy : 0.020509958267211914 nb_pixel_total : 30341 time to create 1 rle with old method : 0.03155088424682617 time for calcul the mask position with numpy : 0.01992344856262207 nb_pixel_total : 10364 time to create 1 rle with old method : 0.010879755020141602 time for calcul the mask position with numpy : 0.02041172981262207 nb_pixel_total : 7654 time to create 1 rle with old method : 0.008028984069824219 time for calcul the mask position with numpy : 0.020635604858398438 nb_pixel_total : 24101 time to create 1 rle with old method : 0.025240421295166016 time for calcul the mask position with numpy : 0.021567344665527344 nb_pixel_total : 177144 time to create 1 rle with new method : 0.9869778156280518 time for calcul the mask position with numpy : 0.024756431579589844 nb_pixel_total : 91595 time to create 1 rle with old method : 0.11152100563049316 time for calcul the mask position with numpy : 0.02658700942993164 nb_pixel_total : 17173 time to create 1 rle with old method : 0.03051304817199707 create new chi : 4.412427186965942 time to delete rle : 0.0020341873168945312 batch 1 Loaded 18 chid ids of type : 3726 Number RLEs to save : 12323 TO DO : save crop sub photo not yet done ! save time : 1.1208608150482178 nb_obj : 6 nb_hashtags : 3 time to prepare the origin masks : 7.225261688232422 time for calcul the mask position with numpy : 0.5995478630065918 nb_pixel_total : 2766803 time to create 1 rle with new method : 1.075516700744629 time for calcul the mask position with numpy : 0.02089548110961914 nb_pixel_total : 169444 time to create 1 rle with new method : 1.2719850540161133 time for calcul the mask position with numpy : 0.019722461700439453 nb_pixel_total : 46055 time to create 1 rle with old method : 0.0462949275970459 time for calcul the mask position with numpy : 0.019667387008666992 nb_pixel_total : 22459 time to create 1 rle with old method : 0.02255988121032715 time for calcul the mask position with numpy : 0.0207059383392334 nb_pixel_total : 174342 time to create 1 rle with new method : 0.9959225654602051 time for calcul the mask position with numpy : 0.02062392234802246 nb_pixel_total : 24121 time to create 1 rle with old method : 0.028499126434326172 time for calcul the mask position with numpy : 2.463210105895996 nb_pixel_total : 3847016 time to create 1 rle with new method : 0.9892973899841309 create new chi : 7.699044942855835 time to delete rle : 0.001119375228881836 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 13090 TO DO : save crop sub photo not yet done ! save time : 1.0043096542358398 nb_obj : 6 nb_hashtags : 4 time to prepare the origin masks : 11.824286460876465 time for calcul the mask position with numpy : 1.2665913105010986 nb_pixel_total : 4244314 time to create 1 rle with new method : 1.3364696502685547 time for calcul the mask position with numpy : 0.021198511123657227 nb_pixel_total : 2693 time to create 1 rle with old method : 0.0027687549591064453 time for calcul the mask position with numpy : 0.019205093383789062 nb_pixel_total : 39290 time to create 1 rle with old method : 0.04095005989074707 time for calcul the mask position with numpy : 0.018855810165405273 nb_pixel_total : 7390 time to create 1 rle with old method : 0.0073697566986083984 time for calcul the mask position with numpy : 0.019049406051635742 nb_pixel_total : 13914 time to create 1 rle with old method : 0.014244556427001953 time for calcul the mask position with numpy : 0.02015209197998047 nb_pixel_total : 95799 time to create 1 rle with old method : 0.10173535346984863 time for calcul the mask position with numpy : 2.876487970352173 nb_pixel_total : 2646840 time to create 1 rle with new method : 1.2984883785247803 create new chi : 7.100344181060791 time to delete rle : 0.0009784698486328125 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 11039 TO DO : save crop sub photo not yet done ! save time : 0.9620983600616455 nb_obj : 19 nb_hashtags : 7 time to prepare the origin masks : 24.118370056152344 time for calcul the mask position with numpy : 0.6968426704406738 nb_pixel_total : 6226289 time to create 1 rle with new method : 0.9480757713317871 time for calcul the mask position with numpy : 0.020433902740478516 nb_pixel_total : 44252 time to create 1 rle with old method : 0.0495302677154541 time for calcul the mask position with numpy : 0.020838260650634766 nb_pixel_total : 4358 time to create 1 rle with old method : 0.004897117614746094 time for calcul the mask position with numpy : 0.01982593536376953 nb_pixel_total : 4641 time to create 1 rle with old method : 0.005232572555541992 time for calcul the mask position with numpy : 0.02024674415588379 nb_pixel_total : 3462 time to create 1 rle with old method : 0.0038781166076660156 time for calcul the mask position with numpy : 0.02025890350341797 nb_pixel_total : 46944 time to create 1 rle with old method : 0.05138254165649414 time for calcul the mask position with numpy : 0.021466970443725586 nb_pixel_total : 131775 time to create 1 rle with old method : 0.13678503036499023 time for calcul the mask position with numpy : 0.021927356719970703 nb_pixel_total : 69552 time to create 1 rle with old method : 0.07181000709533691 time for calcul the mask position with numpy : 0.023623943328857422 nb_pixel_total : 226532 time to create 1 rle with new method : 0.7322366237640381 time for calcul the mask position with numpy : 0.026105642318725586 nb_pixel_total : 37513 time to create 1 rle with old method : 0.04030656814575195 time for calcul the mask position with numpy : 0.021705150604248047 nb_pixel_total : 9581 time to create 1 rle with old method : 0.010045528411865234 time for calcul the mask position with numpy : 0.02026844024658203 nb_pixel_total : 6222 time to create 1 rle with old method : 0.006663322448730469 time for calcul the mask position with numpy : 0.020838499069213867 nb_pixel_total : 1650 time to create 1 rle with old method : 0.0018846988677978516 time for calcul the mask position with numpy : 0.02063751220703125 nb_pixel_total : 17791 time to create 1 rle with old method : 0.018812894821166992 time for calcul the mask position with numpy : 0.021826982498168945 nb_pixel_total : 93423 time to create 1 rle with old method : 0.09584903717041016 time for calcul the mask position with numpy : 0.02048802375793457 nb_pixel_total : 42429 time to create 1 rle with old method : 0.04496169090270996 time for calcul the mask position with numpy : 0.02095317840576172 nb_pixel_total : 55999 time to create 1 rle with old method : 0.05767035484313965 time for calcul the mask position with numpy : 0.020045757293701172 nb_pixel_total : 2377 time to create 1 rle with old method : 0.0025739669799804688 time for calcul the mask position with numpy : 0.0208284854888916 nb_pixel_total : 4403 time to create 1 rle with old method : 0.004511594772338867 time for calcul the mask position with numpy : 0.019882917404174805 nb_pixel_total : 21047 time to create 1 rle with old method : 0.021128177642822266 create new chi : 3.4665114879608154 time to delete rle : 0.0013551712036132812 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 12184 TO DO : save crop sub photo not yet done ! save time : 1.3411779403686523 map_output_result : {1339415827: (0.0, 'Should be the crop_list due to order', 0), 1339415737: (0.0, 'Should be the crop_list due to order', 0), 1339415732: (0.0, 'Should be the crop_list due to order', 0), 1339415700: (0.0, 'Should be the crop_list due to order', 0), 1339415655: (0.0, 'Should be the crop_list due to order', 0), 1339415650: (0.0, 'Should be the crop_list due to order', 0), 1339415646: (0.0, 'Should be the crop_list due to order', 0), 1339322976: (0.0, 'Should be the crop_list due to order', 0), 1339322973: (0.0, 'Should be the crop_list due to order', 0), 1339322946: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1339415827, 1339415737, 1339415732, 1339415700, 1339415655, 1339415650, 1339415646, 1339322976, 1339322973, 1339322946] Looping around the photos to save general results len do output : 10 /1339415827.Didn't retrieve data . /1339415737.Didn't retrieve data . /1339415732.Didn't retrieve data . /1339415700.Didn't retrieve data . /1339415655.Didn't retrieve data . /1339415650.Didn't retrieve data . /1339415646.Didn't retrieve data . /1339322976.Didn't retrieve data . /1339322973.Didn't retrieve data . /1339322946.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415827', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415737', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415732', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415700', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415655', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415650', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415646', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322976', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322973', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322946', None, None, None, None, None, '2611126') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.015382766723632812 save_final save missing photos in datou_result : time spend for datou_step_exec : 180.96667981147766 time spend to save output : 0.016026973724365234 total time spend for step 5 : 180.98270678520203 step6:crop_condition Tue Feb 25 00:47:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 136 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740440858_2772460 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.9861533641815186 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740440863_2772460 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.6260058879852295 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740440870_2772460 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.748762845993042 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740440875_2772460 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.7319769859313965 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740440880_2772460 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.3241772651672363 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740440886_2772460 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.2795109748840332 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339415827, 1339415737, 1339415732, 1339415700, 1339415655, 1339415650, 1339415646, 1339322976, 1339322973, 1339322946] Looping around the photos to save general results len do output : 27 /1339450311Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450312Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450313Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450314Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450315Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450316Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450317Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450318Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450319Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450322Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450323Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450324Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450325Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450326Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450327Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450329Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450330Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450331Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450332Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450333Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450334Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450335Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450336Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450337Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450338Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450339Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339450340Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415827', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415737', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415732', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415700', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415655', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415650', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415646', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322976', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322973', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322946', None, None, None, None, None, '2611126') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 91 time used for this insertion : 0.10681891441345215 save_final save missing photos in datou_result : time spend for datou_step_exec : 35.08407759666443 time spend to save output : 0.1076810359954834 total time spend for step 6 : 35.19175863265991 step7:ventilate_hashtags_in_portfolio Tue Feb 25 00:48:08 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20785558 get user id for portfolio 20785558 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785558 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Teint_Dans_La_Masse','Carton_gris','autre_refus','kraft','metal','mal_croppe','plastique','cartonnette','environnement','flou','Carton_brun','papier')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785558 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Teint_Dans_La_Masse','Carton_gris','autre_refus','kraft','metal','mal_croppe','plastique','cartonnette','environnement','flou','Carton_brun','papier')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785558 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Teint_Dans_La_Masse','Carton_gris','autre_refus','kraft','metal','mal_croppe','plastique','cartonnette','environnement','flou','Carton_brun','papier')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20785608,20785609,20785610,20785611,20785612,20785613,20785614,20785615,20785616,20785617,20785618,20785619?tags=Teint_Dans_La_Masse,Carton_gris,autre_refus,kraft,metal,mal_croppe,plastique,cartonnette,environnement,flou,Carton_brun,papier Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1339415827, 1339415737, 1339415732, 1339415700, 1339415655, 1339415650, 1339415646, 1339322976, 1339322973, 1339322946] Looping around the photos to save general results len do output : 1 /20785558. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415827', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415737', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415732', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415700', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415655', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415650', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415646', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322976', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322973', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322946', None, None, None, None, None, '2611126') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.0162966251373291 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.726109504699707 time spend to save output : 0.016507625579833984 total time spend for step 7 : 0.742617130279541 step8:final Tue Feb 25 00:48:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1339415827: ('0.05545902546010053',), 1339415737: ('0.05545902546010053',), 1339415732: ('0.05545902546010053',), 1339415700: ('0.05545902546010053',), 1339415655: ('0.05545902546010053',), 1339415650: ('0.05545902546010053',), 1339415646: ('0.05545902546010053',), 1339322976: ('0.05545902546010053',), 1339322973: ('0.05545902546010053',), 1339322946: ('0.05545902546010053',)} new output for save of step final : {1339415827: ('0.05545902546010053',), 1339415737: ('0.05545902546010053',), 1339415732: ('0.05545902546010053',), 1339415700: ('0.05545902546010053',), 1339415655: ('0.05545902546010053',), 1339415650: ('0.05545902546010053',), 1339415646: ('0.05545902546010053',), 1339322976: ('0.05545902546010053',), 1339322973: ('0.05545902546010053',), 1339322946: ('0.05545902546010053',)} [1339415827, 1339415737, 1339415732, 1339415700, 1339415655, 1339415650, 1339415646, 1339322976, 1339322973, 1339322946] Looping around the photos to save general results len do output : 10 /1339415827.Didn't retrieve data . /1339415737.Didn't retrieve data . /1339415732.Didn't retrieve data . /1339415700.Didn't retrieve data . /1339415655.Didn't retrieve data . /1339415650.Didn't retrieve data . /1339415646.Didn't retrieve data . /1339322976.Didn't retrieve data . /1339322973.Didn't retrieve data . /1339322946.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415827', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415737', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415732', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415700', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415655', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415650', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415646', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322976', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322973', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322946', None, None, None, None, None, '2611126') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.017912864685058594 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10137057304382324 time spend to save output : 0.018416881561279297 total time spend for step 8 : 0.11978745460510254 step9:velours_tree Tue Feb 25 00:48:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.07831001281738281 time spend to save output : 3.337860107421875e-05 total time spend for step 9 : 0.07834339141845703 step10:send_mail_cod Tue Feb 25 00:48:09 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20785558_25-02-2025_00_48_09.pdf 20785610 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207856101740440889 20785611 imagette207856111740440890 20785612 imagette207856121740440890 20785613 imagette207856131740440890 20785614 change filename to text .change filename to text .change filename to text .imagette207856141740440890 20785615 change filename to text .change filename to text .change filename to text .change filename to text .imagette207856151740440890 20785617 imagette207856171740440890 20785608 change filename to text .change filename to text .change filename to text .change filename to text .imagette207856081740440890 20785609 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207856091740440890 20785618 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette207856181740440891 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20785558 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20785608,20785609,20785610,20785611,20785612,20785613,20785614,20785615,20785616,20785617,20785618,20785619?tags=Teint_Dans_La_Masse,Carton_gris,autre_refus,kraft,metal,mal_croppe,plastique,cartonnette,environnement,flou,Carton_brun,papier your option no_mail is active, we will not send the real mail to your client args[1339415827] : ('0.05545902546010053',) no score found for photo 1339415827 We are sending mail with results at report@fotonower.com args[1339415737] : ('0.05545902546010053',) no score found for photo 1339415737 We are sending mail with results at report@fotonower.com args[1339415732] : ('0.05545902546010053',) no score found for photo 1339415732 We are sending mail with results at report@fotonower.com args[1339415700] : ('0.05545902546010053',) no score found for photo 1339415700 We are sending mail with results at report@fotonower.com args[1339415655] : ('0.05545902546010053',) no score found for photo 1339415655 We are sending mail with results at report@fotonower.com args[1339415650] : ('0.05545902546010053',) no score found for photo 1339415650 We are sending mail with results at report@fotonower.com args[1339415646] : ('0.05545902546010053',) no score found for photo 1339415646 We are sending mail with results at report@fotonower.com args[1339322976] : ('0.05545902546010053',) no score found for photo 1339322976 We are sending mail with results at report@fotonower.com args[1339322973] : ('0.05545902546010053',) no score found for photo 1339322973 We are sending mail with results at report@fotonower.com args[1339322946] : ('0.05545902546010053',) no score found for photo 1339322946 We are sending mail with results at report@fotonower.com refus_total : 0.05545902546010053 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20785558 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1339415737,1339322202,1339322209,1339322212,1339322224,1339322227,1339322231,1339322235,1339322239,1339415732,1339415646,1339322184,1339322943,1339415650,1339322976,1339322711,1339415655,1339415700,1339415827,1339322973) Found this number of photos: 20 begin to download photo : 1339415737 begin to download photo : 1339322227 begin to download photo : 1339415646 begin to download photo : 1339322711 download finish for photo 1339415646 begin to download photo : 1339322184 download finish for photo 1339415737 begin to download photo : 1339322202 download finish for photo 1339322711 begin to download photo : 1339415655 download finish for photo 1339322227 begin to download photo : 1339322231 download finish for photo 1339322184 begin to download photo : 1339322943 download finish for photo 1339415655 begin to download photo : 1339415700 download finish for photo 1339322231 begin to download photo : 1339322235 download finish for photo 1339322202 begin to download photo : 1339322209 download finish for photo 1339322943 begin to download photo : 1339415650 download finish for photo 1339415700 begin to download photo : 1339415827 download finish for photo 1339322235 begin to download photo : 1339322239 download finish for photo 1339322209 begin to download photo : 1339322212 download finish for photo 1339415650 begin to download photo : 1339322976 download finish for photo 1339415827 begin to download photo : 1339322973 download finish for photo 1339322239 begin to download photo : 1339415732 download finish for photo 1339322212 begin to download photo : 1339322224 download finish for photo 1339322976 download finish for photo 1339415732 download finish for photo 1339322973 download finish for photo 1339322224 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785558_25-02-2025_00_48_09.pdf results_Auto_P20785558_25-02-2025_00_48_09.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785558_25-02-2025_00_48_09.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20785558','results_Auto_P20785558_25-02-2025_00_48_09.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785558_25-02-2025_00_48_09.pdf','pdf','','0.7','0.05545902546010053') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1339415827, 1339415737, 1339415732, 1339415700, 1339415655, 1339415650, 1339415646, 1339322976, 1339322973, 1339322946] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415827', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415737', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415732', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415700', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415655', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415650', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415646', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322976', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322973', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322946', None, None, None, None, None, '2611126') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.01616382598876953 save_final save missing photos in datou_result : time spend for datou_step_exec : 5.9691572189331055 time spend to save output : 0.016431331634521484 total time spend for step 10 : 5.985588550567627 step11:split_time_score Tue Feb 25 00:48:15 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('08', 27),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 24022025 20785558 Nombre de photos uploadées : 27 / 23040 (0%) 24022025 20785558 Nombre de photos taguées (types de déchets): 0 / 27 (0%) 24022025 20785558 Nombre de photos taguées (volume) : 0 / 27 (0%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 5.245208740234375e-06 ??????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0013451576232910156 elapsed_time : insert_dashboard_record_day_entry 0.02442002296447754 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785553 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785554 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785555 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785556 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785557 order by id desc limit 1 Qualite : 0.05545902546010053 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20785558_25-02-2025_00_48_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785558 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20785558 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785559 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785560 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785561 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785562 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785563 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785564 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785565 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785566 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785567 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782984 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785568 order by id desc limit 1 Qualite : 0.22267187071827646 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20782986_24-02-2025_22_36_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782986 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782986 AND mptpi.`type`=4200 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782987 order by id desc limit 1 Qualite : 0.12499677790385096 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20782988_24-02-2025_22_23_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782988 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11548 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11556 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11557 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11552 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11552 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11555 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11555 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11549 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11580 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11580 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11551 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11550 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11550 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11559 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11551 doesn't seem to be define in the database( WARNING : type of input 3 of step 11550 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11548 doesn't seem to be define in the database( WARNING : type of input 2 of step 11552 doesn't seem to be define in the database( WARNING : output 1 of step 11548 have datatype=2 whereas input 1 of step 11555 have datatype=7 WARNING : type of output 2 of step 11555 doesn't seem to be define in the database( WARNING : type of input 1 of step 11549 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11551 have datatype=10 whereas input 3 of step 11558 have datatype=6 WARNING : type of input 2 of step 11580 doesn't seem to be define in the database( WARNING : output 1 of step 11549 have datatype=7 whereas input 2 of step 11580 have datatype=None WARNING : type of output 3 of step 11580 doesn't seem to be define in the database( WARNING : type of input 1 of step 11551 doesn't seem to be define in the database( WARNING : output 0 of step 11551 have datatype=10 whereas input 0 of step 11586 have datatype=18 WARNING : type of input 5 of step 11558 doesn't seem to be define in the database( WARNING : output 0 of step 11586 have datatype=11 whereas input 5 of step 11558 have datatype=None WARNING : type of output 1 of step 11556 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : type of output 1 of step 11557 doesn't seem to be define in the database( WARNING : type of input 3 of step 11552 doesn't seem to be define in the database( WARNING : output 0 of step 11555 have datatype=1 whereas input 0 of step 11549 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782988 AND mptpi.`type`=4200 To do Qualite : 0.2106218921467765 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782989 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782989 AND mptpi.`type`=3594 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782990 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782991 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782992 order by id desc limit 1 Qualite : 0.08146381333007933 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20782993_24-02-2025_22_29_15.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782993 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782993 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782994 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782995 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782996 order by id desc limit 1 Qualite : 0.09881400042563439 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20782997_25-02-2025_00_46_14.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782997 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782997 AND mptpi.`type`=4207 To do Qualite : 0.048543972049013485 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20782998_25-02-2025_00_31_10.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782998 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782998 AND mptpi.`type`=4207 To do Qualite : 0.06136040084196714 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20782999_25-02-2025_00_32_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20782999 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11560 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11567 mask_detect have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11567 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11563 crop_condition is not consistent : 4 used against 2 in the step definition ! Step 11563 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11564 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11564 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11565 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11573 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11573 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11568 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11566 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11566 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 1 of step 11560 have datatype=2 whereas input 1 of step 11564 have datatype=7 WARNING : type of output 2 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11565 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11567 doesn't seem to be define in the database( WARNING : type of input 3 of step 11563 doesn't seem to be define in the database( WARNING : type of output 3 of step 11564 doesn't seem to be define in the database( WARNING : type of input 1 of step 11568 doesn't seem to be define in the database( WARNING : type of output 1 of step 11568 doesn't seem to be define in the database( WARNING : type of input 3 of step 11566 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11570 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of input 2 of step 11563 doesn't seem to be define in the database( WARNING : output 0 of step 11569 have datatype=6 whereas input 2 of step 11563 have datatype=None WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11570 doesn't seem to be define in the database( WARNING : type of output 2 of step 11560 doesn't seem to be define in the database( WARNING : type of input 1 of step 11569 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11568 have datatype=10 whereas input 3 of step 11571 have datatype=6 WARNING : type of input 2 of step 11573 doesn't seem to be define in the database( WARNING : output 1 of step 11565 have datatype=7 whereas input 2 of step 11573 have datatype=None WARNING : type of output 3 of step 11573 doesn't seem to be define in the database( WARNING : type of input 3 of step 11568 doesn't seem to be define in the database( WARNING : output 0 of step 11568 have datatype=10 whereas input 0 of step 11587 have datatype=18 WARNING : type of input 5 of step 11571 doesn't seem to be define in the database( WARNING : output 0 of step 11587 have datatype=11 whereas input 5 of step 11571 have datatype=None WARNING : output 0 of step 11564 have datatype=1 whereas input 0 of step 11565 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20782999 AND mptpi.`type`=3327 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785569 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785570 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785571 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785572 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785573 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785574 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785575 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785576 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785577 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785578 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20785579 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783011 order by id desc limit 1 Qualite : 0.20103672889149876 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20783012_24-02-2025_23_45_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783012 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20783012 AND mptpi.`type`=3594 To do Qualite : 0.052401843008330756 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20783013_24-02-2025_22_36_50.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783013 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20783013 AND mptpi.`type`=3726 To do Qualite : 0.10145267883062205 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20783014_24-02-2025_23_25_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783014 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20783014 AND mptpi.`type`=4207 To do Qualite : 0.056568829757248966 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20783015_24-02-2025_22_49_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20783015 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20783015 AND mptpi.`type`=4207 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'24022025': {'nb_upload': 27, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1339415827, 1339415737, 1339415732, 1339415700, 1339415655, 1339415650, 1339415646, 1339322976, 1339322973, 1339322946] Looping around the photos to save general results len do output : 1 /20785558Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415827', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415737', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415732', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415700', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415655', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415650', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339415646', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322976', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322973', None, None, None, None, None, '2611126') ('3459', None, None, None, None, None, None, None, '2611126') ('3459', '20785558', '1339322946', None, None, None, None, None, '2611126') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.013694524765014648 save_final save missing photos in datou_result : time spend for datou_step_exec : 28.228410720825195 time spend to save output : 0.013885736465454102 total time spend for step 11 : 28.24229645729065 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 129.31user 197.33system 6:18.65elapsed 86%CPU (0avgtext+0avgdata 4314404maxresident)k 2273592inputs+117704outputs (9553major+12117211minor)pagefaults 0swaps