python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 3459 -l 10 ' -s datou_current_3459 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4157988 load datou : 3459 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? id de la photo (peut être local ou global) was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) load thcls load THCL from format json or kwargs add thcl : 2456 in CacheModelConfig load THCL from format json or kwargs add thcl : 2868 in CacheModelConfig load THCL from format json or kwargs add thcl : 2896 in CacheModelConfig load pdts add pdt : 4999 in CacheModelConfig add pdt : 5288 in CacheModelConfig add pdt : 5309 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 we have a portfolio with more photos than limit : 22>10 please execute split_portfolio.py -i 20699439 -l 10 size over we load limit photo not treated list_input_json: [] Current got : datou_id : 3459, datou_cur_ids : ['2601855'] with mtr_portfolio_ids : ['20699439'] and first list_photo_ids : [] new path : /proc/4157988/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, thcl, merge_mask_thcl_custom, rle_unique_nms_with_priority, crop_condition, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 10 ; length of list_pids : 10 ; length of list_args : 10 time to download the photos : 1.8247404098510742 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Fri Feb 21 00:12:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3183 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-21 00:12:35.138522: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-21 00:12:35.175054: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-21 00:12:35.177437: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f7d14000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-21 00:12:35.177496: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-21 00:12:35.183859: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-21 00:12:35.448049: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4383ed70 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-21 00:12:35.448103: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-21 00:12:35.449495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-21 00:12:35.449946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-21 00:12:35.453065: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-21 00:12:35.471655: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-21 00:12:35.472138: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-21 00:12:35.505801: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-21 00:12:35.511513: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-21 00:12:35.565269: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-21 00:12:35.566553: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-21 00:12:35.567020: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-21 00:12:35.567693: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-21 00:12:35.567712: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-21 00:12:35.567724: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-21 00:12:35.569149: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2731 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-21 00:12:35.900674: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-21 00:12:35.900770: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-21 00:12:35.900791: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-21 00:12:35.900810: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-21 00:12:35.900828: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-21 00:12:35.900846: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-21 00:12:35.900863: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-21 00:12:35.900881: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-21 00:12:35.901877: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-21 00:12:35.902943: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-21 00:12:35.902986: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-21 00:12:35.903002: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-21 00:12:35.903015: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-21 00:12:35.903030: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-21 00:12:35.903043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-21 00:12:35.903057: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-21 00:12:35.903071: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-21 00:12:35.903897: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-21 00:12:35.903925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-21 00:12:35.903933: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-21 00:12:35.903941: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-21 00:12:35.904812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2731 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2896 thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5309, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 16384, 25088, 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 5, 10, 19, 20, 46), datetime.datetime(2021, 5, 10, 19, 20, 46)) {'thcl': {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'], 'list_hashtags_csv': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'svm_hashtag_type_desc': 5309, 'photo_desc_type': 5309, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_convoyeur_qualipapia_nantes_poly_100521_1 NUM_CLASSES 10 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_convoyeur_qualipapia_nantes_poly_100521_1 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-21 00:12:48.675868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-21 00:12:48.856081: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-21 00:12:50.634784: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.634865: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.635478: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.635497: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.642508: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.642529: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.643077: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.643095: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.649579: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.649601: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.650185: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.650201: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 466.56MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.681534: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.681591: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.682520: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.682563: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.689049: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.689101: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.689990: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.690015: W tensorflow/core/common_runtime/bfc_allocator.cc:245] Allocator (GPU_0_bfc) ran out of memory trying to allocate 243.25MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2025-02-21 00:12:50.724330: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.724956: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.726833: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.727449: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.771441: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.772072: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.774312: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.774910: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.805598: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.806228: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.807997: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.808597: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.815180: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.815785: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.817630: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.818169: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.824906: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.825458: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.827214: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.827805: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.857191: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.857860: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.858494: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.859045: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.862541: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.863145: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.879700: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.880767: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.881915: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.883031: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.896060: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.896869: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.897643: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.898416: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.902930: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.903541: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.908471: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.909058: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.921691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.922337: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.926691: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.927371: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.927969: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.928577: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.929736: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.930795: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.941325: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.941917: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.942491: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.943077: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.943671: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.944262: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.959489: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:50.960102: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.007303: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.007942: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.015886: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.016481: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.032812: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.033574: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.034310: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.035075: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.039249: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.039842: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.040427: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.041010: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.042005: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.042024: W tensorflow/core/kernels/gpu_utils.cc:49] Failed to allocate memory for convolution redzone checking; skipping this check. This is benign and only means that we won't check cudnn for out-of-bounds reads and writes. This message will only be printed once. 2025-02-21 00:12:51.052338: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.052929: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.062207: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.062817: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.063434: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.064022: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.064618: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory 2025-02-21 00:12:51.065203: I tensorflow/stream_executor/cuda/cuda_driver.cc:763] failed to allocate 2.17G (2328756224 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY: out of memory local folder : /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1 /data/models_weight/learn_convoyeur_qualipapia_nantes_poly_100521_1/mask_model.h5 size_local : 256031040 size in s3 : 256031040 create time local : 2021-08-09 05:45:48 create time in s3 : 2021-08-06 18:59:51 mask_model.h5 already exist and didn't need to update list_images length : 10 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 9 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 17 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 23 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 5 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 25 NEW PHOTO Processing 1 images image shape: (2160, 3264, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 18) min: 0.00000 max: 3264.00000 nb d'objets trouves : 8 Detection mask done ! Trying to reset tf kernel 4158636 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 1990 tf kernel not reseted sub process len(results) : 10 len(list_Values) 0 None max_time_sub_proc : 3600 parent process len(results) : 10 len(list_Values) 0 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 3183 list_Values should be empty [] To do loadFromThcl(), then load ParamDescType : thcl2896 Catched exception ! Connect or reconnect ! thcls : [{'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'}] thcl {'id': 2896, 'mtr_user_id': 31, 'name': 'learn_convoyeur_qualipapia_nantes_poly_100521_1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,carton_brun,carton_gris,cartonnette,kraft,autre_refus,metal,plastique,teint_dans_la_masse,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3663, 'photo_desc_type': 5309, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5309 ['background', 'carton_brun', 'carton_gris', 'cartonnette', 'kraft', 'autre_refus', 'metal', 'plastique', 'teint_dans_la_masse', 'environnement'] time for calcul the mask position with numpy : 0.00048232078552246094 nb_pixel_total : 19173 time to create 1 rle with old method : 0.02127552032470703 length of segment : 140 time for calcul the mask position with numpy : 0.0005850791931152344 nb_pixel_total : 22818 time to create 1 rle with old method : 0.026589155197143555 length of segment : 218 time for calcul the mask position with numpy : 0.003551959991455078 nb_pixel_total : 156291 time to create 1 rle with new method : 0.010403871536254883 length of segment : 779 time for calcul the mask position with numpy : 0.0019538402557373047 nb_pixel_total : 130423 time to create 1 rle with old method : 0.15590214729309082 length of segment : 407 time for calcul the mask position with numpy : 0.0002808570861816406 nb_pixel_total : 11335 time to create 1 rle with old method : 0.013456344604492188 length of segment : 129 time for calcul the mask position with numpy : 0.0002295970916748047 nb_pixel_total : 8394 time to create 1 rle with old method : 0.009495019912719727 length of segment : 145 time for calcul the mask position with numpy : 0.000579833984375 nb_pixel_total : 34038 time to create 1 rle with old method : 0.03718376159667969 length of segment : 222 time for calcul the mask position with numpy : 0.0007433891296386719 nb_pixel_total : 41207 time to create 1 rle with old method : 0.043985605239868164 length of segment : 373 time for calcul the mask position with numpy : 0.003977060317993164 nb_pixel_total : 160537 time to create 1 rle with new method : 0.028028249740600586 length of segment : 612 time for calcul the mask position with numpy : 0.0002582073211669922 nb_pixel_total : 10390 time to create 1 rle with old method : 0.012746810913085938 length of segment : 113 time for calcul the mask position with numpy : 0.00042939186096191406 nb_pixel_total : 16271 time to create 1 rle with old method : 0.01968216896057129 length of segment : 177 time for calcul the mask position with numpy : 0.00021409988403320312 nb_pixel_total : 11627 time to create 1 rle with old method : 0.012713432312011719 length of segment : 122 time for calcul the mask position with numpy : 0.00022530555725097656 nb_pixel_total : 10980 time to create 1 rle with old method : 0.012967109680175781 length of segment : 192 time for calcul the mask position with numpy : 7.581710815429688e-05 nb_pixel_total : 1707 time to create 1 rle with old method : 0.0021309852600097656 length of segment : 82 time for calcul the mask position with numpy : 0.0015802383422851562 nb_pixel_total : 84853 time to create 1 rle with old method : 0.09152936935424805 length of segment : 349 time for calcul the mask position with numpy : 0.0002899169921875 nb_pixel_total : 13331 time to create 1 rle with old method : 0.014816522598266602 length of segment : 128 time for calcul the mask position with numpy : 0.0015566349029541016 nb_pixel_total : 94634 time to create 1 rle with old method : 0.10256743431091309 length of segment : 341 time for calcul the mask position with numpy : 0.00016617774963378906 nb_pixel_total : 3557 time to create 1 rle with old method : 0.004352092742919922 length of segment : 132 time for calcul the mask position with numpy : 0.0032045841217041016 nb_pixel_total : 183132 time to create 1 rle with new method : 0.013206720352172852 length of segment : 681 time for calcul the mask position with numpy : 0.0002613067626953125 nb_pixel_total : 9132 time to create 1 rle with old method : 0.010747194290161133 length of segment : 118 time for calcul the mask position with numpy : 0.00030612945556640625 nb_pixel_total : 12341 time to create 1 rle with old method : 0.013978242874145508 length of segment : 115 time for calcul the mask position with numpy : 0.0002269744873046875 nb_pixel_total : 8616 time to create 1 rle with old method : 0.00985407829284668 length of segment : 107 time for calcul the mask position with numpy : 0.0011501312255859375 nb_pixel_total : 59379 time to create 1 rle with old method : 0.06588554382324219 length of segment : 364 time for calcul the mask position with numpy : 0.001402139663696289 nb_pixel_total : 67605 time to create 1 rle with old method : 0.07107806205749512 length of segment : 316 time for calcul the mask position with numpy : 0.0007781982421875 nb_pixel_total : 43257 time to create 1 rle with old method : 0.050818681716918945 length of segment : 348 time for calcul the mask position with numpy : 0.004594564437866211 nb_pixel_total : 213756 time to create 1 rle with new method : 0.01648998260498047 length of segment : 542 time for calcul the mask position with numpy : 0.0020704269409179688 nb_pixel_total : 121753 time to create 1 rle with old method : 0.12883472442626953 length of segment : 605 time for calcul the mask position with numpy : 0.0023834705352783203 nb_pixel_total : 120616 time to create 1 rle with old method : 0.12877559661865234 length of segment : 442 time for calcul the mask position with numpy : 0.004281520843505859 nb_pixel_total : 135975 time to create 1 rle with old method : 0.16402578353881836 length of segment : 406 time for calcul the mask position with numpy : 0.0006659030914306641 nb_pixel_total : 41571 time to create 1 rle with old method : 0.047547340393066406 length of segment : 277 time for calcul the mask position with numpy : 0.002470731735229492 nb_pixel_total : 133258 time to create 1 rle with old method : 0.15384387969970703 length of segment : 416 time for calcul the mask position with numpy : 0.0012824535369873047 nb_pixel_total : 47740 time to create 1 rle with old method : 0.061846017837524414 length of segment : 359 time for calcul the mask position with numpy : 0.0013501644134521484 nb_pixel_total : 65854 time to create 1 rle with old method : 0.07660150527954102 length of segment : 381 time for calcul the mask position with numpy : 0.0010528564453125 nb_pixel_total : 28293 time to create 1 rle with old method : 0.03211402893066406 length of segment : 376 time for calcul the mask position with numpy : 0.0006084442138671875 nb_pixel_total : 1486 time to create 1 rle with old method : 0.002384662628173828 length of segment : 86 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 1223 time to create 1 rle with old method : 0.001588582992553711 length of segment : 48 time for calcul the mask position with numpy : 0.0002498626708984375 nb_pixel_total : 11611 time to create 1 rle with old method : 0.01363682746887207 length of segment : 159 time for calcul the mask position with numpy : 0.0002505779266357422 nb_pixel_total : 10421 time to create 1 rle with old method : 0.011823654174804688 length of segment : 149 time for calcul the mask position with numpy : 0.00012803077697753906 nb_pixel_total : 1199 time to create 1 rle with old method : 0.0015418529510498047 length of segment : 68 time for calcul the mask position with numpy : 0.0003769397735595703 nb_pixel_total : 8553 time to create 1 rle with old method : 0.010634422302246094 length of segment : 132 time for calcul the mask position with numpy : 0.004845619201660156 nb_pixel_total : 211433 time to create 1 rle with new method : 0.01141810417175293 length of segment : 695 time for calcul the mask position with numpy : 0.0013682842254638672 nb_pixel_total : 56649 time to create 1 rle with old method : 0.05985069274902344 length of segment : 468 time for calcul the mask position with numpy : 0.0009613037109375 nb_pixel_total : 20189 time to create 1 rle with old method : 0.027668476104736328 length of segment : 252 time for calcul the mask position with numpy : 0.0005061626434326172 nb_pixel_total : 14021 time to create 1 rle with old method : 0.01774764060974121 length of segment : 139 time for calcul the mask position with numpy : 0.006458282470703125 nb_pixel_total : 150752 time to create 1 rle with new method : 0.022889137268066406 length of segment : 680 time for calcul the mask position with numpy : 0.0009253025054931641 nb_pixel_total : 57042 time to create 1 rle with old method : 0.0669407844543457 length of segment : 370 time for calcul the mask position with numpy : 0.0007033348083496094 nb_pixel_total : 12957 time to create 1 rle with old method : 0.01602959632873535 length of segment : 236 time for calcul the mask position with numpy : 0.0008296966552734375 nb_pixel_total : 24847 time to create 1 rle with old method : 0.03032827377319336 length of segment : 201 time for calcul the mask position with numpy : 0.005266427993774414 nb_pixel_total : 202269 time to create 1 rle with new method : 0.02588033676147461 length of segment : 800 time for calcul the mask position with numpy : 0.0029439926147460938 nb_pixel_total : 99701 time to create 1 rle with old method : 0.10761737823486328 length of segment : 522 time for calcul the mask position with numpy : 0.0003864765167236328 nb_pixel_total : 6703 time to create 1 rle with old method : 0.007269144058227539 length of segment : 132 time for calcul the mask position with numpy : 0.0025832653045654297 nb_pixel_total : 80688 time to create 1 rle with old method : 0.08540463447570801 length of segment : 554 time for calcul the mask position with numpy : 0.0011990070343017578 nb_pixel_total : 44614 time to create 1 rle with old method : 0.07073664665222168 length of segment : 273 time for calcul the mask position with numpy : 0.002097606658935547 nb_pixel_total : 69711 time to create 1 rle with old method : 0.07343387603759766 length of segment : 486 time for calcul the mask position with numpy : 0.0019745826721191406 nb_pixel_total : 52654 time to create 1 rle with old method : 0.05808568000793457 length of segment : 398 time for calcul the mask position with numpy : 0.0006730556488037109 nb_pixel_total : 19361 time to create 1 rle with old method : 0.021039724349975586 length of segment : 135 time for calcul the mask position with numpy : 0.0002982616424560547 nb_pixel_total : 2728 time to create 1 rle with old method : 0.003542661666870117 length of segment : 112 time for calcul the mask position with numpy : 0.00016427040100097656 nb_pixel_total : 1577 time to create 1 rle with old method : 0.0020933151245117188 length of segment : 62 time for calcul the mask position with numpy : 0.0030553340911865234 nb_pixel_total : 122554 time to create 1 rle with old method : 0.13010740280151367 length of segment : 404 time for calcul the mask position with numpy : 0.04342007637023926 nb_pixel_total : 875820 time to create 1 rle with new method : 0.46156883239746094 length of segment : 1407 time for calcul the mask position with numpy : 0.006508588790893555 nb_pixel_total : 185486 time to create 1 rle with new method : 0.015172243118286133 length of segment : 472 time for calcul the mask position with numpy : 0.005703926086425781 nb_pixel_total : 224620 time to create 1 rle with new method : 0.008652448654174805 length of segment : 719 time for calcul the mask position with numpy : 0.0015146732330322266 nb_pixel_total : 42906 time to create 1 rle with old method : 0.04719877243041992 length of segment : 298 time for calcul the mask position with numpy : 0.00027751922607421875 nb_pixel_total : 8975 time to create 1 rle with old method : 0.014852285385131836 length of segment : 85 time for calcul the mask position with numpy : 0.0001621246337890625 nb_pixel_total : 1655 time to create 1 rle with old method : 0.0029757022857666016 length of segment : 42 time for calcul the mask position with numpy : 0.004157543182373047 nb_pixel_total : 142923 time to create 1 rle with old method : 0.16109037399291992 length of segment : 444 time for calcul the mask position with numpy : 0.0011663436889648438 nb_pixel_total : 37911 time to create 1 rle with old method : 0.04436469078063965 length of segment : 329 time for calcul the mask position with numpy : 0.0008945465087890625 nb_pixel_total : 46146 time to create 1 rle with old method : 0.05312538146972656 length of segment : 303 time for calcul the mask position with numpy : 0.0011396408081054688 nb_pixel_total : 36526 time to create 1 rle with old method : 0.04195141792297363 length of segment : 215 time for calcul the mask position with numpy : 0.0026068687438964844 nb_pixel_total : 85828 time to create 1 rle with old method : 0.09430313110351562 length of segment : 614 time for calcul the mask position with numpy : 0.49030494689941406 nb_pixel_total : 4162451 time to create 1 rle with new method : 1.2010133266448975 length of segment : 4067 time for calcul the mask position with numpy : 9.1552734375e-05 nb_pixel_total : 1800 time to create 1 rle with old method : 0.0022737979888916016 length of segment : 48 time for calcul the mask position with numpy : 0.0022041797637939453 nb_pixel_total : 182275 time to create 1 rle with new method : 0.0035703182220458984 length of segment : 393 time for calcul the mask position with numpy : 0.0028228759765625 nb_pixel_total : 115587 time to create 1 rle with old method : 0.12543821334838867 length of segment : 713 time for calcul the mask position with numpy : 0.00018835067749023438 nb_pixel_total : 8493 time to create 1 rle with old method : 0.010226964950561523 length of segment : 129 time for calcul the mask position with numpy : 0.0007288455963134766 nb_pixel_total : 47820 time to create 1 rle with old method : 0.05549478530883789 length of segment : 270 time for calcul the mask position with numpy : 0.5183038711547852 nb_pixel_total : 4022157 time to create 1 rle with new method : 0.7158010005950928 length of segment : 3116 time for calcul the mask position with numpy : 0.0056591033935546875 nb_pixel_total : 401410 time to create 1 rle with new method : 0.008835554122924805 length of segment : 610 time for calcul the mask position with numpy : 0.007226228713989258 nb_pixel_total : 426888 time to create 1 rle with new method : 0.8960070610046387 length of segment : 855 time for calcul the mask position with numpy : 0.00428318977355957 nb_pixel_total : 174669 time to create 1 rle with new method : 0.008700847625732422 length of segment : 646 time for calcul the mask position with numpy : 0.0006301403045654297 nb_pixel_total : 18073 time to create 1 rle with old method : 0.02052593231201172 length of segment : 198 time for calcul the mask position with numpy : 0.0031270980834960938 nb_pixel_total : 123536 time to create 1 rle with old method : 0.15965962409973145 length of segment : 491 time for calcul the mask position with numpy : 0.0003120899200439453 nb_pixel_total : 6428 time to create 1 rle with old method : 0.009341716766357422 length of segment : 93 time for calcul the mask position with numpy : 0.0007100105285644531 nb_pixel_total : 23438 time to create 1 rle with old method : 0.02471017837524414 length of segment : 201 time for calcul the mask position with numpy : 0.0006284713745117188 nb_pixel_total : 22678 time to create 1 rle with old method : 0.026112079620361328 length of segment : 185 time for calcul the mask position with numpy : 0.001073598861694336 nb_pixel_total : 45981 time to create 1 rle with old method : 0.06241202354431152 length of segment : 225 time for calcul the mask position with numpy : 0.0013344287872314453 nb_pixel_total : 40744 time to create 1 rle with old method : 0.051132917404174805 length of segment : 290 time for calcul the mask position with numpy : 0.0013036727905273438 nb_pixel_total : 51907 time to create 1 rle with old method : 0.059178829193115234 length of segment : 336 time for calcul the mask position with numpy : 0.0011491775512695312 nb_pixel_total : 32989 time to create 1 rle with old method : 0.04941749572753906 length of segment : 190 time for calcul the mask position with numpy : 0.0007698535919189453 nb_pixel_total : 23527 time to create 1 rle with old method : 0.02721095085144043 length of segment : 209 time for calcul the mask position with numpy : 0.0007650852203369141 nb_pixel_total : 24968 time to create 1 rle with old method : 0.028943300247192383 length of segment : 161 time for calcul the mask position with numpy : 0.0001881122589111328 nb_pixel_total : 3419 time to create 1 rle with old method : 0.004019737243652344 length of segment : 99 time for calcul the mask position with numpy : 0.003087282180786133 nb_pixel_total : 156960 time to create 1 rle with new method : 0.004774570465087891 length of segment : 371 time for calcul the mask position with numpy : 0.003382444381713867 nb_pixel_total : 125695 time to create 1 rle with old method : 0.14275717735290527 length of segment : 497 time for calcul the mask position with numpy : 0.0017669200897216797 nb_pixel_total : 41184 time to create 1 rle with old method : 0.05035138130187988 length of segment : 305 time for calcul the mask position with numpy : 0.0002593994140625 nb_pixel_total : 9285 time to create 1 rle with old method : 0.010911941528320312 length of segment : 86 time for calcul the mask position with numpy : 0.0004935264587402344 nb_pixel_total : 5038 time to create 1 rle with old method : 0.006352424621582031 length of segment : 184 time for calcul the mask position with numpy : 0.0005733966827392578 nb_pixel_total : 11850 time to create 1 rle with old method : 0.01454615592956543 length of segment : 192 time for calcul the mask position with numpy : 0.0013391971588134766 nb_pixel_total : 42826 time to create 1 rle with old method : 0.050943613052368164 length of segment : 451 time for calcul the mask position with numpy : 0.00028777122497558594 nb_pixel_total : 11586 time to create 1 rle with old method : 0.013116598129272461 length of segment : 132 time for calcul the mask position with numpy : 0.00041794776916503906 nb_pixel_total : 13163 time to create 1 rle with old method : 0.015628337860107422 length of segment : 205 time for calcul the mask position with numpy : 0.00033283233642578125 nb_pixel_total : 9942 time to create 1 rle with old method : 0.011827707290649414 length of segment : 127 time for calcul the mask position with numpy : 0.0016982555389404297 nb_pixel_total : 63115 time to create 1 rle with old method : 0.0684659481048584 length of segment : 385 time for calcul the mask position with numpy : 0.001988649368286133 nb_pixel_total : 66481 time to create 1 rle with old method : 0.09503984451293945 length of segment : 229 time for calcul the mask position with numpy : 0.0022325515747070312 nb_pixel_total : 78053 time to create 1 rle with old method : 0.08678698539733887 length of segment : 396 time for calcul the mask position with numpy : 0.0010128021240234375 nb_pixel_total : 64239 time to create 1 rle with old method : 0.07380867004394531 length of segment : 211 time for calcul the mask position with numpy : 0.002794981002807617 nb_pixel_total : 110888 time to create 1 rle with old method : 0.1277785301208496 length of segment : 585 time for calcul the mask position with numpy : 0.0002982616424560547 nb_pixel_total : 10093 time to create 1 rle with old method : 0.011521100997924805 length of segment : 135 time for calcul the mask position with numpy : 0.002553224563598633 nb_pixel_total : 102423 time to create 1 rle with old method : 0.11112475395202637 length of segment : 512 time for calcul the mask position with numpy : 0.003322601318359375 nb_pixel_total : 79348 time to create 1 rle with old method : 0.08467483520507812 length of segment : 613 time for calcul the mask position with numpy : 0.0014677047729492188 nb_pixel_total : 65513 time to create 1 rle with old method : 0.06998562812805176 length of segment : 392 time for calcul the mask position with numpy : 0.001931905746459961 nb_pixel_total : 103154 time to create 1 rle with old method : 0.12253284454345703 length of segment : 297 time for calcul the mask position with numpy : 0.0008881092071533203 nb_pixel_total : 19271 time to create 1 rle with old method : 0.021992921829223633 length of segment : 311 time for calcul the mask position with numpy : 0.00043272972106933594 nb_pixel_total : 3915 time to create 1 rle with old method : 0.006102561950683594 length of segment : 67 time for calcul the mask position with numpy : 0.0031180381774902344 nb_pixel_total : 127712 time to create 1 rle with old method : 0.14011478424072266 length of segment : 566 time for calcul the mask position with numpy : 0.0010404586791992188 nb_pixel_total : 63943 time to create 1 rle with old method : 0.06914520263671875 length of segment : 255 time for calcul the mask position with numpy : 0.00026106834411621094 nb_pixel_total : 12815 time to create 1 rle with old method : 0.015298843383789062 length of segment : 108 time for calcul the mask position with numpy : 0.0002913475036621094 nb_pixel_total : 6655 time to create 1 rle with old method : 0.0077211856842041016 length of segment : 126 time for calcul the mask position with numpy : 0.0007538795471191406 nb_pixel_total : 33761 time to create 1 rle with old method : 0.036092519760131836 length of segment : 326 time for calcul the mask position with numpy : 0.0007603168487548828 nb_pixel_total : 47155 time to create 1 rle with old method : 0.05087685585021973 length of segment : 351 time for calcul the mask position with numpy : 0.0016262531280517578 nb_pixel_total : 74968 time to create 1 rle with old method : 0.08354020118713379 length of segment : 366 time for calcul the mask position with numpy : 0.016551494598388672 nb_pixel_total : 577350 time to create 1 rle with new method : 0.40224671363830566 length of segment : 1717 time for calcul the mask position with numpy : 0.09006619453430176 nb_pixel_total : 2230653 time to create 1 rle with new method : 0.6389241218566895 length of segment : 4044 time for calcul the mask position with numpy : 0.017031431198120117 nb_pixel_total : 990048 time to create 1 rle with new method : 0.07037210464477539 length of segment : 1793 time for calcul the mask position with numpy : 0.00012040138244628906 nb_pixel_total : 2730 time to create 1 rle with old method : 0.00450444221496582 length of segment : 85 time for calcul the mask position with numpy : 0.004120349884033203 nb_pixel_total : 342171 time to create 1 rle with new method : 0.009148120880126953 length of segment : 678 time for calcul the mask position with numpy : 0.004762887954711914 nb_pixel_total : 346528 time to create 1 rle with new method : 0.06149578094482422 length of segment : 757 time spent for convertir_results : 23.839189767837524 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Number saved : None batch 1 Loaded 127 chid ids of type : 3663 Number RLEs to save : 54383 save missing photos in datou_result : time spend for datou_step_exec : 75.45247983932495 time spend to save output : 4.828648805618286 total time spend for step 1 : 80.28112864494324 step2:crop_condition Fri Feb 21 00:13:51 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure Loading chi in step crop with photo_hashtag_type : 3663 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 127 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 15 About to insert : list_path_to_insert length 15 new photo from crops ! we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 11 About to insert : list_path_to_insert length 11 new photo from crops ! we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 46 About to insert : list_path_to_insert length 46 new photo from crops ! we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 7 About to insert : list_path_to_insert length 7 new photo from crops ! we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 we have both polygon and rles Next one ! we have both polygon and rles Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1338690108, 1338690104, 1338690094, 1338690091, 1338690088, 1338690085, 1338690078, 1338640318, 1338640277, 1338640275] Looping around the photos to save general results len do output : 83 /-3681201659Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201660Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201670Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201671Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201677Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201681Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201687Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201679Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201700Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201705Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201713Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201722Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201734Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201764Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201766Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201666Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201673Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201708Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201711Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201699Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201709Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201724Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201749Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201773Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201775Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201777Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201731Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201774Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201662Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201667Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201661Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201675Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201683Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201693Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201695Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201685Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201686Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201688Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201691Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201684Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201692Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201689Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201690Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201703Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201706Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201710Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201701Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201719Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201717Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201727Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201721Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201728Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201732Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201736Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201748Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201751Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201742Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201759Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201752Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201747Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201757Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201745Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201740Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201769Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201761Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201765Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201779Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201768Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201771Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201762Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201770Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201763Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201778Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201785Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201669Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201678Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201680Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201704Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201743Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201753Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201754Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201682Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /-3681201726Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690108', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690104', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690094', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690091', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690088', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690085', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690078', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640318', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640277', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640275', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 259 time used for this insertion : 0.04164552688598633 save_final save missing photos in datou_result : time spend for datou_step_exec : 39.19409656524658 time spend to save output : 0.044115543365478516 total time spend for step 2 : 39.23821210861206 step3:thcl Fri Feb 21 00:14:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! nombre de thcls : 2 we are using the classfication for multi_thcl [2456, 2868] time to import caffe and check if the image exist : 0.004331350326538086 time to convert the images to numpy array : 0.033492088317871094 time to import caffe and check if the image exist : 0.007655143737792969 time to convert the images to numpy array : 0.10492992401123047 time to import caffe and check if the image exist : 0.007973432540893555 time to convert the images to numpy array : 1.4520595073699951 time to import caffe and check if the image exist : 0.010265588760375977 time to convert the images to numpy array : 1.4654698371887207 time to import caffe and check if the image exist : 0.011545658111572266 time to convert the images to numpy array : 1.4663705825805664 time to import caffe and check if the image exist : 0.010373115539550781 time to convert the images to numpy array : 1.4718444347381592 time to import caffe and check if the image exist : 0.011267900466918945 time to convert the images to numpy array : 1.4845995903015137 time to import caffe and check if the image exist : 0.011630773544311523 time to convert the images to numpy array : 1.4992756843566895 time to import caffe and check if the image exist : 0.01315617561340332 time to convert the images to numpy array : 1.4982326030731201 time to import caffe and check if the image exist : 0.006365299224853516 time to convert the images to numpy array : 1.5369677543640137 total time to convert the images to numpy array : 1.8930673599243164 list photo_ids error: [] list photo_ids correct : [-3681201682, -3681201726, -3681201659, -3681201660, -3681201670, -3681201671, -3681201677, -3681201681, -3681201687, -3681201679, -3681201700, -3681201705, -3681201713, -3681201722, -3681201734, -3681201764, -3681201766, -3681201666, -3681201673, -3681201708, -3681201711, -3681201699, -3681201709, -3681201724, -3681201749, -3681201773, -3681201775, -3681201777, -3681201731, -3681201748, -3681201751, -3681201742, -3681201759, -3681201752, -3681201747, -3681201757, -3681201745, -3681201740, -3681201769, -3681201761, -3681201765, -3681201779, -3681201768, -3681201771, -3681201762, -3681201770, -3681201763, -3681201778, -3681201785, -3681201669, -3681201678, -3681201680, -3681201704, -3681201743, -3681201753, -3681201754, -3681201774, -3681201662, -3681201667, -3681201661, -3681201675, -3681201683, -3681201693, -3681201695, -3681201685, -3681201686, -3681201688, -3681201691, -3681201684, -3681201692, -3681201689, -3681201690, -3681201703, -3681201706, -3681201710, -3681201701, -3681201719, -3681201717, -3681201727, -3681201721, -3681201728, -3681201732, -3681201736] number of photos to traite : 83 try to delete the photos incorrect in DB tagging for thcl : 2456 To do loadFromThcl(), then load ParamDescType : thcl2456 thcls : [{'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'}] thcl {'id': 2456, 'mtr_user_id': 31, 'name': 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'papier,refus', 'svm_portfolios_learning': '3028087,3028251', 'photo_hashtag_type': 3049, 'photo_desc_type': 4999, 'type_classification': 'caffe', 'hashtag_id_list': '492668766,538914404'} Update svm_hashtag_type_desc : 4999 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) To loadFromThcl() : net_4999 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 3183 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4999, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 16384, 25088, 'learn_qualipapia_papier_refus_from_vlg_data_aug', 'res5b', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 10, 23, 14, 27, 22), datetime.datetime(2020, 10, 23, 14, 27, 22)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_qualipapia_papier_refus_from_vlg_data_aug Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_qualipapia_papier_refus_from_vlg_data_aug model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel size_local : 44972172 size in s3 : 44972172 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 caffemodel already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt size_local : 17311 size in s3 : 17311 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:51 mean.npy already exist and didn't need to update /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/synset_words.txt size_local : 57 size in s3 : 57 create time local : 2021-08-09 05:55:48 create time in s3 : 2021-08-06 19:28:49 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/deploy.prototxt caffemodel_filename : /data/models_weight/learn_qualipapia_papier_refus_from_vlg_data_aug/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 2645 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 0.9063587188720703 time used to do the prediction : 0.20541095733642578 we don't save the descriptors for this thcl 2456 tagging for thcl : 2868 To do loadFromThcl(), then load ParamDescType : thcl2868 thcls : [{'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2868, 'mtr_user_id': 31, 'name': 'learn_papier_nantes_300421', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3752117,3752118,3752123,3752106,3752116,3752124,3752119,3581575,3486029,3752122', 'photo_hashtag_type': 3632, 'photo_desc_type': 5288, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5288 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) To loadFromThcl() : net_5288 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 2163 wait 20 seconds l 3637 free memory gpu now : 2163 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5288, 'learn_papier_nantes_300421', 512, 512, 'learn_papier_nantes_300421', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 30, 17, 9, 41), datetime.datetime(2021, 4, 30, 17, 9, 41)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_papier_nantes_300421 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_papier_nantes_300421 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_papier_nantes_300421 /data/models_weight/learn_papier_nantes_300421/caffemodel size_local : 44791983 size in s3 : 44791983 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:13 caffemodel already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/deploy.prototxt size_local : 17255 size in s3 : 17255 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:12 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-08-09 05:55:59 create time in s3 : 2021-08-06 19:22:14 mean.npy already exist and didn't need to update /data/models_weight/learn_papier_nantes_300421/synset_words.txt size_local : 331 size in s3 : 331 create time local : 2021-08-09 05:56:00 create time in s3 : 2021-08-06 19:22:12 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_papier_nantes_300421/deploy.prototxt caffemodel_filename : /data/models_weight/learn_papier_nantes_300421/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 1769 wait 20 seconds l 3637 free memory gpu now : 1769 max_wait_temp : 1 max_wait : 0 dict_keys(['prob']) time used to do the prepocess of the images : 1.009641408920288 time used to do the prediction : 0.19421052932739258 we don't save the descriptors for this thcl 2868 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos test new format of the output of the step_thcl begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 0 time used for this insertion : 1.0251998901367188e-05 save missing photos in datou_result : time spend for datou_step_exec : 51.13090229034424 time spend to save output : 0.0006029605865478516 total time spend for step 3 : 51.131505250930786 step4:merge_mask_thcl_custom Fri Feb 21 00:15:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step merge_mask_thcl_custom batch 1 Loaded 127 chid ids of type : 3663 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++As expected we have just one thcl present End of step merge_mask_thcl_custom Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : merge_mask_thcl_custom we use saveGeneral [1338690108, 1338690104, 1338690094, 1338690091, 1338690088, 1338690085, 1338690078, 1338640318, 1338640277, 1338640275] Looping around the photos to save general results len do output : 10 /1338690108Didn't retrieve data .Didn't retrieve data . /1338690104Didn't retrieve data .Didn't retrieve data . /1338690094Didn't retrieve data .Didn't retrieve data . /1338690091Didn't retrieve data .Didn't retrieve data . /1338690088Didn't retrieve data .Didn't retrieve data . /1338690085Didn't retrieve data .Didn't retrieve data . /1338690078Didn't retrieve data .Didn't retrieve data . /1338640318Didn't retrieve data .Didn't retrieve data . /1338640277Didn't retrieve data .Didn't retrieve data . /1338640275Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690108', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690104', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690094', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690091', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690088', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690085', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690078', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640318', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640277', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640275', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.017363309860229492 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.07060050964355469 time spend to save output : 0.01807546615600586 total time spend for step 4 : 0.08867597579956055 step5:rle_unique_nms_with_priority Fri Feb 21 00:15:21 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms nb_obj : 9 nb_hashtags : 6 time to prepare the origin masks : 6.509345293045044 time for calcul the mask position with numpy : 0.3387153148651123 nb_pixel_total : 6467444 time to create 1 rle with new method : 0.3414030075073242 time for calcul the mask position with numpy : 0.027468204498291016 nb_pixel_total : 155834 time to create 1 rle with new method : 0.6474089622497559 time for calcul the mask position with numpy : 0.027791738510131836 nb_pixel_total : 33875 time to create 1 rle with old method : 0.036749839782714844 time for calcul the mask position with numpy : 0.025368928909301758 nb_pixel_total : 8334 time to create 1 rle with old method : 0.009429931640625 time for calcul the mask position with numpy : 0.025659799575805664 nb_pixel_total : 11265 time to create 1 rle with old method : 0.012131452560424805 time for calcul the mask position with numpy : 0.026937007904052734 nb_pixel_total : 41138 time to create 1 rle with old method : 0.04465150833129883 time for calcul the mask position with numpy : 0.025924205780029297 nb_pixel_total : 160305 time to create 1 rle with new method : 0.6005487442016602 time for calcul the mask position with numpy : 0.027892351150512695 nb_pixel_total : 19107 time to create 1 rle with old method : 0.0212249755859375 time for calcul the mask position with numpy : 0.025686025619506836 nb_pixel_total : 130176 time to create 1 rle with old method : 0.16674065589904785 time for calcul the mask position with numpy : 0.02670574188232422 nb_pixel_total : 22762 time to create 1 rle with old method : 0.025206327438354492 create new chi : 2.5677571296691895 time to delete rle : 0.014693498611450195 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 8228 TO DO : save crop sub photo not yet done ! save time : 7.114647388458252 nb_obj : 9 nb_hashtags : 5 time to prepare the origin masks : 10.631915092468262 time for calcul the mask position with numpy : 0.4608955383300781 nb_pixel_total : 6886782 time to create 1 rle with new method : 0.35343360900878906 time for calcul the mask position with numpy : 0.022548198699951172 nb_pixel_total : 16091 time to create 1 rle with old method : 0.017863988876342773 time for calcul the mask position with numpy : 0.022277355194091797 nb_pixel_total : 3531 time to create 1 rle with old method : 0.0044405460357666016 time for calcul the mask position with numpy : 0.02347087860107422 nb_pixel_total : 6887 time to create 1 rle with old method : 0.008865118026733398 time for calcul the mask position with numpy : 0.025495529174804688 nb_pixel_total : 89235 time to create 1 rle with old method : 0.09871530532836914 time for calcul the mask position with numpy : 0.022412538528442383 nb_pixel_total : 13235 time to create 1 rle with old method : 0.014381647109985352 time for calcul the mask position with numpy : 0.02152705192565918 nb_pixel_total : 1705 time to create 1 rle with old method : 0.0020599365234375 time for calcul the mask position with numpy : 0.020794153213500977 nb_pixel_total : 11575 time to create 1 rle with old method : 0.013926506042480469 time for calcul the mask position with numpy : 0.021199464797973633 nb_pixel_total : 10935 time to create 1 rle with old method : 0.012764930725097656 time for calcul the mask position with numpy : 0.02181267738342285 nb_pixel_total : 10264 time to create 1 rle with old method : 0.011238813400268555 create new chi : 1.2382628917694092 time to delete rle : 0.00024175643920898438 batch 1 Loaded 10 chid ids of type : 3726 Number RLEs to save : 5178 TO DO : save crop sub photo not yet done ! save time : 0.36570119857788086 nb_obj : 21 nb_hashtags : 7 time to prepare the origin masks : 30.018784999847412 time for calcul the mask position with numpy : 0.8269388675689697 nb_pixel_total : 5926386 time to create 1 rle with new method : 1.3126394748687744 time for calcul the mask position with numpy : 0.034165143966674805 nb_pixel_total : 43217 time to create 1 rle with old method : 0.046247005462646484 time for calcul the mask position with numpy : 0.03373265266418457 nb_pixel_total : 1188 time to create 1 rle with old method : 0.0014142990112304688 time for calcul the mask position with numpy : 0.0373687744140625 nb_pixel_total : 120678 time to create 1 rle with old method : 0.13365817070007324 time for calcul the mask position with numpy : 0.03507280349731445 nb_pixel_total : 1215 time to create 1 rle with old method : 0.0017881393432617188 time for calcul the mask position with numpy : 0.03856086730957031 nb_pixel_total : 135749 time to create 1 rle with old method : 0.17919301986694336 time for calcul the mask position with numpy : 0.021301746368408203 nb_pixel_total : 10390 time to create 1 rle with old method : 0.011470794677734375 time for calcul the mask position with numpy : 0.022110700607299805 nb_pixel_total : 6617 time to create 1 rle with old method : 0.008060932159423828 time for calcul the mask position with numpy : 0.024246692657470703 nb_pixel_total : 59292 time to create 1 rle with old method : 0.06475138664245605 time for calcul the mask position with numpy : 0.025149822235107422 nb_pixel_total : 1219 time to create 1 rle with old method : 0.0014786720275878906 time for calcul the mask position with numpy : 0.0218353271484375 nb_pixel_total : 9080 time to create 1 rle with old method : 0.010365962982177734 time for calcul the mask position with numpy : 0.02234172821044922 nb_pixel_total : 10573 time to create 1 rle with old method : 0.011545658111572266 time for calcul the mask position with numpy : 0.021524429321289062 nb_pixel_total : 8611 time to create 1 rle with old method : 0.009891271591186523 time for calcul the mask position with numpy : 0.022818803787231445 nb_pixel_total : 182647 time to create 1 rle with new method : 0.9863841533660889 time for calcul the mask position with numpy : 0.042513370513916016 nb_pixel_total : 47642 time to create 1 rle with old method : 0.05397748947143555 time for calcul the mask position with numpy : 0.040381669998168945 nb_pixel_total : 121654 time to create 1 rle with old method : 0.1391758918762207 time for calcul the mask position with numpy : 0.034786224365234375 nb_pixel_total : 41523 time to create 1 rle with old method : 0.04737401008605957 time for calcul the mask position with numpy : 0.03398275375366211 nb_pixel_total : 12262 time to create 1 rle with old method : 0.014151811599731445 time for calcul the mask position with numpy : 0.034226179122924805 nb_pixel_total : 67380 time to create 1 rle with old method : 0.07734894752502441 time for calcul the mask position with numpy : 0.031723976135253906 nb_pixel_total : 213279 time to create 1 rle with new method : 1.9044411182403564 time for calcul the mask position with numpy : 0.03359651565551758 nb_pixel_total : 28159 time to create 1 rle with old method : 0.03108501434326172 time for calcul the mask position with numpy : 0.0317385196685791 nb_pixel_total : 1479 time to create 1 rle with old method : 0.0018265247344970703 create new chi : 6.601831674575806 time to delete rle : 0.033060550689697266 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 14217 TO DO : save crop sub photo not yet done ! save time : 1.1336300373077393 nb_obj : 14 nb_hashtags : 7 time to prepare the origin masks : 16.594865322113037 time for calcul the mask position with numpy : 1.0257787704467773 nb_pixel_total : 6266739 time to create 1 rle with new method : 0.7309310436248779 time for calcul the mask position with numpy : 0.036400556564331055 nb_pixel_total : 12910 time to create 1 rle with old method : 0.014587879180908203 time for calcul the mask position with numpy : 0.03776144981384277 nb_pixel_total : 208327 time to create 1 rle with new method : 0.527026891708374 time for calcul the mask position with numpy : 0.03611874580383301 nb_pixel_total : 95708 time to create 1 rle with old method : 0.10943841934204102 time for calcul the mask position with numpy : 0.022848844528198242 nb_pixel_total : 20102 time to create 1 rle with old method : 0.02393054962158203 time for calcul the mask position with numpy : 0.024201154708862305 nb_pixel_total : 3941 time to create 1 rle with old method : 0.005105733871459961 time for calcul the mask position with numpy : 0.02489614486694336 nb_pixel_total : 57017 time to create 1 rle with old method : 0.06350183486938477 time for calcul the mask position with numpy : 0.02517390251159668 nb_pixel_total : 6686 time to create 1 rle with old method : 0.007480144500732422 time for calcul the mask position with numpy : 0.025008201599121094 nb_pixel_total : 13957 time to create 1 rle with old method : 0.020922422409057617 time for calcul the mask position with numpy : 0.025371313095092773 nb_pixel_total : 57289 time to create 1 rle with old method : 0.0768575668334961 time for calcul the mask position with numpy : 0.02497076988220215 nb_pixel_total : 149397 time to create 1 rle with old method : 0.1694962978363037 time for calcul the mask position with numpy : 0.022288084030151367 nb_pixel_total : 24764 time to create 1 rle with old method : 0.027576923370361328 time for calcul the mask position with numpy : 0.021876096725463867 nb_pixel_total : 44522 time to create 1 rle with old method : 0.06027483940124512 time for calcul the mask position with numpy : 0.02477097511291504 nb_pixel_total : 80410 time to create 1 rle with old method : 0.10303449630737305 time for calcul the mask position with numpy : 0.022463083267211914 nb_pixel_total : 8471 time to create 1 rle with old method : 0.010092973709106445 create new chi : 3.4164836406707764 time to delete rle : 0.0006759166717529297 batch 1 Loaded 15 chid ids of type : 3726 Number RLEs to save : 12081 TO DO : save crop sub photo not yet done ! save time : 0.7351090908050537 nb_obj : 16 nb_hashtags : 6 time to prepare the origin masks : 13.928053617477417 time for calcul the mask position with numpy : 0.25824618339538574 nb_pixel_total : 5185074 time to create 1 rle with new method : 1.5472869873046875 time for calcul the mask position with numpy : 0.03940320014953613 nb_pixel_total : 42869 time to create 1 rle with old method : 0.04383349418640137 time for calcul the mask position with numpy : 0.023914575576782227 nb_pixel_total : 36456 time to create 1 rle with old method : 0.03918790817260742 time for calcul the mask position with numpy : 0.024813413619995117 nb_pixel_total : 1633 time to create 1 rle with old method : 0.0019230842590332031 time for calcul the mask position with numpy : 0.030041217803955078 nb_pixel_total : 142596 time to create 1 rle with old method : 0.1587679386138916 time for calcul the mask position with numpy : 0.022926807403564453 nb_pixel_total : 46095 time to create 1 rle with old method : 0.04745888710021973 time for calcul the mask position with numpy : 0.025456666946411133 nb_pixel_total : 184852 time to create 1 rle with new method : 0.7750308513641357 time for calcul the mask position with numpy : 0.023567914962768555 nb_pixel_total : 8892 time to create 1 rle with old method : 0.009656429290771484 time for calcul the mask position with numpy : 0.02390265464782715 nb_pixel_total : 34112 time to create 1 rle with old method : 0.03603982925415039 time for calcul the mask position with numpy : 0.02620983123779297 nb_pixel_total : 122404 time to create 1 rle with old method : 0.12876129150390625 time for calcul the mask position with numpy : 0.025945425033569336 nb_pixel_total : 224696 time to create 1 rle with new method : 0.5129125118255615 time for calcul the mask position with numpy : 0.035742998123168945 nb_pixel_total : 873836 time to create 1 rle with new method : 0.628960132598877 time for calcul the mask position with numpy : 0.022823572158813477 nb_pixel_total : 1565 time to create 1 rle with old method : 0.0018982887268066406 time for calcul the mask position with numpy : 0.023975372314453125 nb_pixel_total : 53435 time to create 1 rle with old method : 0.0545041561126709 time for calcul the mask position with numpy : 0.02639293670654297 nb_pixel_total : 2721 time to create 1 rle with old method : 0.002989053726196289 time for calcul the mask position with numpy : 0.02823328971862793 nb_pixel_total : 19218 time to create 1 rle with old method : 0.021167993545532227 time for calcul the mask position with numpy : 0.026637554168701172 nb_pixel_total : 69786 time to create 1 rle with old method : 0.07539486885070801 create new chi : 4.900543451309204 time to delete rle : 0.0006906986236572266 batch 1 Loaded 17 chid ids of type : 3726 Number RLEs to save : 14041 TO DO : save crop sub photo not yet done ! save time : 0.9697380065917969 nb_obj : 5 nb_hashtags : 3 time to prepare the origin masks : 7.236586809158325 time for calcul the mask position with numpy : 0.044634103775024414 nb_pixel_total : 2597898 time to create 1 rle with new method : 0.9988093376159668 time for calcul the mask position with numpy : 0.026234865188598633 nb_pixel_total : 109949 time to create 1 rle with old method : 0.1425762176513672 time for calcul the mask position with numpy : 0.024979114532470703 nb_pixel_total : 166523 time to create 1 rle with new method : 0.7394134998321533 time for calcul the mask position with numpy : 0.020470380783081055 nb_pixel_total : 8452 time to create 1 rle with old method : 0.009357213973999023 time for calcul the mask position with numpy : 0.020329713821411133 nb_pixel_total : 1792 time to create 1 rle with old method : 0.0019648075103759766 time for calcul the mask position with numpy : 0.26891398429870605 nb_pixel_total : 4165626 time to create 1 rle with new method : 1.002640962600708 create new chi : 3.3867764472961426 time to delete rle : 0.0007932186126708984 batch 1 Loaded 6 chid ids of type : 3726 Number RLEs to save : 12457 TO DO : save crop sub photo not yet done ! save time : 3.299811601638794 nb_obj : 4 nb_hashtags : 3 time to prepare the origin masks : 5.810902118682861 time for calcul the mask position with numpy : 0.12169146537780762 nb_pixel_total : 2831577 time to create 1 rle with new method : 0.33167243003845215 time for calcul the mask position with numpy : 0.022330045700073242 nb_pixel_total : 38123 time to create 1 rle with old method : 0.04429507255554199 time for calcul the mask position with numpy : 0.022966623306274414 nb_pixel_total : 123565 time to create 1 rle with old method : 0.13695263862609863 time for calcul the mask position with numpy : 0.0220334529876709 nb_pixel_total : 47702 time to create 1 rle with old method : 0.05164051055908203 time for calcul the mask position with numpy : 0.19245362281799316 nb_pixel_total : 4009273 time to create 1 rle with new method : 1.4999439716339111 create new chi : 2.5150389671325684 time to delete rle : 0.00074005126953125 batch 1 Loaded 5 chid ids of type : 3726 Number RLEs to save : 10337 TO DO : save crop sub photo not yet done ! save time : 2.563538074493408 nb_obj : 21 nb_hashtags : 5 time to prepare the origin masks : 23.59080696105957 time for calcul the mask position with numpy : 0.753471851348877 nb_pixel_total : 6061082 time to create 1 rle with new method : 0.9954783916473389 time for calcul the mask position with numpy : 0.04347085952758789 nb_pixel_total : 125798 time to create 1 rle with old method : 0.24344754219055176 time for calcul the mask position with numpy : 0.04009413719177246 nb_pixel_total : 32904 time to create 1 rle with old method : 0.04820728302001953 time for calcul the mask position with numpy : 0.03873085975646973 nb_pixel_total : 3558 time to create 1 rle with old method : 0.0042724609375 time for calcul the mask position with numpy : 0.051046133041381836 nb_pixel_total : 24906 time to create 1 rle with old method : 0.03780078887939453 time for calcul the mask position with numpy : 0.04824399948120117 nb_pixel_total : 41134 time to create 1 rle with old method : 0.04613828659057617 time for calcul the mask position with numpy : 0.03873920440673828 nb_pixel_total : 40526 time to create 1 rle with old method : 0.045099735260009766 time for calcul the mask position with numpy : 0.04027295112609863 nb_pixel_total : 123257 time to create 1 rle with old method : 0.14433646202087402 time for calcul the mask position with numpy : 0.03694725036621094 nb_pixel_total : 5015 time to create 1 rle with old method : 0.005949735641479492 time for calcul the mask position with numpy : 0.03657245635986328 nb_pixel_total : 13111 time to create 1 rle with old method : 0.01804375648498535 time for calcul the mask position with numpy : 0.03630352020263672 nb_pixel_total : 9233 time to create 1 rle with old method : 0.01042318344116211 time for calcul the mask position with numpy : 0.034095048904418945 nb_pixel_total : 23500 time to create 1 rle with old method : 0.041295766830444336 time for calcul the mask position with numpy : 0.03362846374511719 nb_pixel_total : 3408 time to create 1 rle with old method : 0.006107330322265625 time for calcul the mask position with numpy : 0.03235769271850586 nb_pixel_total : 22607 time to create 1 rle with old method : 0.0406341552734375 time for calcul the mask position with numpy : 0.03254532814025879 nb_pixel_total : 156805 time to create 1 rle with new method : 1.3272459506988525 time for calcul the mask position with numpy : 0.02231574058532715 nb_pixel_total : 23370 time to create 1 rle with old method : 0.025671958923339844 time for calcul the mask position with numpy : 0.0250396728515625 nb_pixel_total : 52601 time to create 1 rle with old method : 0.055304765701293945 time for calcul the mask position with numpy : 0.02306079864501953 nb_pixel_total : 45725 time to create 1 rle with old method : 0.05408763885498047 time for calcul the mask position with numpy : 0.025684118270874023 nb_pixel_total : 6405 time to create 1 rle with old method : 0.007845401763916016 time for calcul the mask position with numpy : 0.022171497344970703 nb_pixel_total : 18038 time to create 1 rle with old method : 0.019756078720092773 time for calcul the mask position with numpy : 0.02367377281188965 nb_pixel_total : 42759 time to create 1 rle with old method : 0.05024123191833496 time for calcul the mask position with numpy : 0.023675203323364258 nb_pixel_total : 174498 time to create 1 rle with new method : 0.6272127628326416 create new chi : 5.415283203125 time to delete rle : 0.0009164810180664062 batch 1 Loaded 22 chid ids of type : 3726 Number RLEs to save : 13175 TO DO : save crop sub photo not yet done ! save time : 5.010132789611816 nb_obj : 19 nb_hashtags : 8 time to prepare the origin masks : 19.15799045562744 time for calcul the mask position with numpy : 0.6202361583709717 nb_pixel_total : 6079617 time to create 1 rle with new method : 0.7750759124755859 time for calcul the mask position with numpy : 0.02949070930480957 nb_pixel_total : 103115 time to create 1 rle with old method : 0.11585330963134766 time for calcul the mask position with numpy : 0.028578758239746094 nb_pixel_total : 12404 time to create 1 rle with old method : 0.016187429428100586 time for calcul the mask position with numpy : 0.033612966537475586 nb_pixel_total : 63047 time to create 1 rle with old method : 0.06775426864624023 time for calcul the mask position with numpy : 0.025643587112426758 nb_pixel_total : 2594 time to create 1 rle with old method : 0.0036199092864990234 time for calcul the mask position with numpy : 0.030701160430908203 nb_pixel_total : 65411 time to create 1 rle with old method : 0.07269525527954102 time for calcul the mask position with numpy : 0.024826765060424805 nb_pixel_total : 19195 time to create 1 rle with old method : 0.020889997482299805 time for calcul the mask position with numpy : 0.023535490036010742 nb_pixel_total : 3863 time to create 1 rle with old method : 0.004261970520019531 time for calcul the mask position with numpy : 0.02438497543334961 nb_pixel_total : 127798 time to create 1 rle with old method : 0.1392381191253662 time for calcul the mask position with numpy : 0.03656578063964844 nb_pixel_total : 15714 time to create 1 rle with old method : 0.02249884605407715 time for calcul the mask position with numpy : 0.029393911361694336 nb_pixel_total : 13346 time to create 1 rle with old method : 0.017779111862182617 time for calcul the mask position with numpy : 0.03216195106506348 nb_pixel_total : 63668 time to create 1 rle with old method : 0.06832385063171387 time for calcul the mask position with numpy : 0.027577638626098633 nb_pixel_total : 98714 time to create 1 rle with old method : 0.10974860191345215 time for calcul the mask position with numpy : 0.029969215393066406 nb_pixel_total : 63940 time to create 1 rle with old method : 0.09073686599731445 time for calcul the mask position with numpy : 0.034362077713012695 nb_pixel_total : 10069 time to create 1 rle with old method : 0.011528730392456055 time for calcul the mask position with numpy : 0.02891373634338379 nb_pixel_total : 33736 time to create 1 rle with old method : 0.03692269325256348 time for calcul the mask position with numpy : 0.031514644622802734 nb_pixel_total : 74773 time to create 1 rle with old method : 0.08465433120727539 time for calcul the mask position with numpy : 0.028555631637573242 nb_pixel_total : 110646 time to create 1 rle with old method : 0.12133598327636719 time for calcul the mask position with numpy : 0.02493429183959961 nb_pixel_total : 9899 time to create 1 rle with old method : 0.01095128059387207 time for calcul the mask position with numpy : 0.029491901397705078 nb_pixel_total : 78691 time to create 1 rle with old method : 0.09859967231750488 create new chi : 3.111851215362549 time to delete rle : 0.0008981227874755859 batch 1 Loaded 20 chid ids of type : 3726 Number RLEs to save : 14178 TO DO : save crop sub photo not yet done ! save time : 1.0227687358856201 nb_obj : 6 nb_hashtags : 2 time to prepare the origin masks : 6.857441663742065 time for calcul the mask position with numpy : 0.36031007766723633 nb_pixel_total : 3393328 time to create 1 rle with new method : 0.9029998779296875 time for calcul the mask position with numpy : 0.04294896125793457 nb_pixel_total : 104294 time to create 1 rle with old method : 0.11977958679199219 time for calcul the mask position with numpy : 0.03561043739318848 nb_pixel_total : 2126 time to create 1 rle with old method : 0.002521038055419922 time for calcul the mask position with numpy : 0.03902840614318848 nb_pixel_total : 424336 time to create 1 rle with new method : 1.819035291671753 time for calcul the mask position with numpy : 0.06256794929504395 nb_pixel_total : 2207615 time to create 1 rle with new method : 0.37390685081481934 time for calcul the mask position with numpy : 0.04012441635131836 nb_pixel_total : 572373 time to create 1 rle with new method : 0.7992100715637207 time for calcul the mask position with numpy : 0.03982377052307129 nb_pixel_total : 346168 time to create 1 rle with new method : 0.886063814163208 create new chi : 5.665492296218872 time to delete rle : 0.0010554790496826172 batch 1 Loaded 7 chid ids of type : 3726 Number RLEs to save : 18185 TO DO : save crop sub photo not yet done ! save time : 2.5032732486724854 map_output_result : {1338690108: (0.0, 'Should be the crop_list due to order', 0), 1338690104: (0.0, 'Should be the crop_list due to order', 0), 1338690094: (0.0, 'Should be the crop_list due to order', 0), 1338690091: (0.0, 'Should be the crop_list due to order', 0), 1338690088: (0.0, 'Should be the crop_list due to order', 0), 1338690085: (0.0, 'Should be the crop_list due to order', 0), 1338690078: (0.0, 'Should be the crop_list due to order', 0), 1338640318: (0.0, 'Should be the crop_list due to order', 0), 1338640277: (0.0, 'Should be the crop_list due to order', 0), 1338640275: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1338690108, 1338690104, 1338690094, 1338690091, 1338690088, 1338690085, 1338690078, 1338640318, 1338640277, 1338640275] Looping around the photos to save general results len do output : 10 /1338690108.Didn't retrieve data . /1338690104.Didn't retrieve data . /1338690094.Didn't retrieve data . /1338690091.Didn't retrieve data . /1338690088.Didn't retrieve data . /1338690085.Didn't retrieve data . /1338690078.Didn't retrieve data . /1338640318.Didn't retrieve data . /1338640277.Didn't retrieve data . /1338640275.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690108', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690104', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690094', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690091', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690088', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690085', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690078', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640318', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640277', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640275', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.01616835594177246 save_final save missing photos in datou_result : time spend for datou_step_exec : 205.51224088668823 time spend to save output : 0.02191019058227539 total time spend for step 5 : 205.5341510772705 step6:crop_condition Fri Feb 21 00:18:47 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 3726 Loading chi in step crop for list_pids : 10 ! batch 1 Loaded 134 chid ids of type : 3726 begin to crop the class : teint_dans_la_masse param for this class : {'min_score': 0.7} filtre for class : teint_dans_la_masse hashtag_id of this class : 2107752385 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 8 About to insert : list_path_to_insert length 8 new photo from crops ! About to upload 8 photos upload in portfolio : 4869462 init cache_photo without model_param we have 8 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740093532_4157988 we have uploaded 8 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.51513409614563 we have finished the crop for the class : teint_dans_la_masse begin to crop the class : autre_refus param for this class : {'min_score': 0.5} filtre for class : autre_refus hashtag_id of this class : 2107752406 Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 5 About to insert : list_path_to_insert length 5 new photo from crops ! About to upload 5 photos upload in portfolio : 4869462 init cache_photo without model_param we have 5 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740093539_4157988 we have uploaded 5 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.995800018310547 we have finished the crop for the class : autre_refus begin to crop the class : carton_gris param for this class : {'min_score': 0.5} filtre for class : carton_gris hashtag_id of this class : 2107753020 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740093546_4157988 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.2271387577056885 we have finished the crop for the class : carton_gris begin to crop the class : cartonnette param for this class : {'min_score': 0.5} filtre for class : cartonnette hashtag_id of this class : 702398920 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740093551_4157988 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.31533145904541 we have finished the crop for the class : cartonnette begin to crop the class : carton_brun param for this class : {'min_score': 0.7} filtre for class : carton_brun hashtag_id of this class : 2107753024 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 6 About to insert : list_path_to_insert length 6 new photo from crops ! About to upload 6 photos upload in portfolio : 4869462 init cache_photo without model_param we have 6 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740093557_4157988 we have uploaded 6 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.9772875308990479 we have finished the crop for the class : carton_brun begin to crop the class : plastique param for this class : {'min_score': 0.5} filtre for class : plastique hashtag_id of this class : 492725882 Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 3 About to insert : list_path_to_insert length 3 new photo from crops ! About to upload 3 photos upload in portfolio : 4869462 init cache_photo without model_param we have 3 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740093562_4157988 we have uploaded 3 photos in the portfolio 4869462 time of upload the photos Elapsed time : 1.5377693176269531 we have finished the crop for the class : plastique begin to crop the class : kraft param for this class : {'min_score': 0.5} filtre for class : kraft hashtag_id of this class : 493202403 begin to crop the class : metal param for this class : {'min_score': 0.5} filtre for class : metal hashtag_id of this class : 492628673 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1338690108, 1338690104, 1338690094, 1338690091, 1338690088, 1338690085, 1338690078, 1338640318, 1338640277, 1338640275] Looping around the photos to save general results len do output : 34 /1338695034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695081Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695114Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338695120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690108', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690104', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690094', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690091', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690088', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690085', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690078', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640318', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640277', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640275', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 112 time used for this insertion : 0.039426326751708984 save_final save missing photos in datou_result : time spend for datou_step_exec : 38.836960315704346 time spend to save output : 0.04128623008728027 total time spend for step 6 : 38.878246545791626 step7:ventilate_hashtags_in_portfolio Fri Feb 21 00:19:26 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! To do loadFromThcl(), then load ParamDescType : thcl2725 thcls : [{'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'}] thcl {'id': 2725, 'mtr_user_id': 31, 'name': 'learn_qualipapia_rle_210302_2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton_brun,Carton_gris,Teint_Dans_La_Masse,autre_refus,cartonnette,environnement,kraft,metal,papier,plastique', 'svm_portfolios_learning': '3460440,3460441,3460446,3460434,3460439,3467416,3460442,3460443,3486028,3460445', 'photo_hashtag_type': 3410, 'photo_desc_type': 5186, 'type_classification': 'caffe', 'hashtag_id_list': '2107753024,2107753020,2107752385,2107752406,702398920,493012381,493202403,492628673,492668766,492725882'} Update svm_hashtag_type_desc : 5186 Iterating over portfolio : 20699439 get user id for portfolio 20699439 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699439 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','plastique','Carton_gris','mal_croppe','Teint_Dans_La_Masse','environnement','flou','Carton_brun','metal','papier','kraft','cartonnette')) AND mptpi.`min_score`=0.5 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699439 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','plastique','Carton_gris','mal_croppe','Teint_Dans_La_Masse','environnement','flou','Carton_brun','metal','papier','kraft','cartonnette')) AND mptpi.`min_score`=0.5 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699439 AND mptpi.`type`=3726 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('autre_refus','plastique','Carton_gris','mal_croppe','Teint_Dans_La_Masse','environnement','flou','Carton_brun','metal','papier','kraft','cartonnette')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://www.fotonower.com/velours/20699694,20699695,20699696,20699697,20699698,20699699,20699700,20699701,20699702,20699703,20699704,20699705?tags=autre_refus,plastique,Carton_gris,mal_croppe,Teint_Dans_La_Masse,environnement,flou,Carton_brun,metal,papier,kraft,cartonnette Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1338690108, 1338690104, 1338690094, 1338690091, 1338690088, 1338690085, 1338690078, 1338640318, 1338640277, 1338640275] Looping around the photos to save general results len do output : 1 /20699439. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690108', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690104', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690094', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690091', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690088', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690085', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690078', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640318', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640277', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640275', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.016892671585083008 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.8166844844818115 time spend to save output : 0.017259836196899414 total time spend for step 7 : 1.833944320678711 step8:final Fri Feb 21 00:19:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1338690108: ('0.7519938324117397',), 1338690104: ('0.7519938324117397',), 1338690094: ('0.7519938324117397',), 1338690091: ('0.7519938324117397',), 1338690088: ('0.7519938324117397',), 1338690085: ('0.7519938324117397',), 1338690078: ('0.7519938324117397',), 1338640318: ('0.7519938324117397',), 1338640277: ('0.7519938324117397',), 1338640275: ('0.7519938324117397',)} new output for save of step final : {1338690108: ('0.7519938324117397',), 1338690104: ('0.7519938324117397',), 1338690094: ('0.7519938324117397',), 1338690091: ('0.7519938324117397',), 1338690088: ('0.7519938324117397',), 1338690085: ('0.7519938324117397',), 1338690078: ('0.7519938324117397',), 1338640318: ('0.7519938324117397',), 1338640277: ('0.7519938324117397',), 1338640275: ('0.7519938324117397',)} [1338690108, 1338690104, 1338690094, 1338690091, 1338690088, 1338690085, 1338690078, 1338640318, 1338640277, 1338640275] Looping around the photos to save general results len do output : 10 /1338690108.Didn't retrieve data . /1338690104.Didn't retrieve data . /1338690094.Didn't retrieve data . /1338690091.Didn't retrieve data . /1338690088.Didn't retrieve data . /1338690085.Didn't retrieve data . /1338690078.Didn't retrieve data . /1338640318.Didn't retrieve data . /1338640277.Didn't retrieve data . /1338640275.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690108', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690104', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690094', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690091', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690088', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690085', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690078', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640318', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640277', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640275', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 30 time used for this insertion : 0.014061450958251953 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10614585876464844 time spend to save output : 0.014885425567626953 total time spend for step 8 : 0.12103128433227539 step9:velours_tree Fri Feb 21 00:19:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure can't find the photo_desc_type Inside saveOutput : final : False verbose : 0 ouput is None No outpout to save, returning out of save general time spend for datou_step_exec : 0.1943359375 time spend to save output : 8.0108642578125e-05 total time spend for step 9 : 0.19441604614257812 step10:send_mail_cod Fri Feb 21 00:19:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Auto_P20699439_21-02-2025_00_19_28.pdf 20699694 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206996941740093568 20699695 change filename to text .change filename to text .change filename to text .imagette206996951740093568 20699697 imagette206996971740093569 20699700 imagette206997001740093569 20699702 imagette206997021740093569 20699704 imagette206997041740093569 20699705 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206997051740093569 20699696 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206996961740093569 20699698 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206996981740093569 20699701 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206997011740093570 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20699439 and hashtag_type = 3726 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20699694,20699695,20699696,20699697,20699698,20699699,20699700,20699701,20699702,20699703,20699704,20699705?tags=autre_refus,plastique,Carton_gris,mal_croppe,Teint_Dans_La_Masse,environnement,flou,Carton_brun,metal,papier,kraft,cartonnette your option no_mail is active, we will not send the real mail to your client args[1338690108] : ('0.7519938324117397',) no score found for photo 1338690108 We are sending mail with results at report@fotonower.com args[1338690104] : ('0.7519938324117397',) no score found for photo 1338690104 We are sending mail with results at report@fotonower.com args[1338690094] : ('0.7519938324117397',) no score found for photo 1338690094 We are sending mail with results at report@fotonower.com args[1338690091] : ('0.7519938324117397',) no score found for photo 1338690091 We are sending mail with results at report@fotonower.com args[1338690088] : ('0.7519938324117397',) no score found for photo 1338690088 We are sending mail with results at report@fotonower.com args[1338690085] : ('0.7519938324117397',) no score found for photo 1338690085 We are sending mail with results at report@fotonower.com args[1338690078] : ('0.7519938324117397',) no score found for photo 1338690078 We are sending mail with results at report@fotonower.com args[1338640318] : ('0.7519938324117397',) no score found for photo 1338640318 We are sending mail with results at report@fotonower.com args[1338640277] : ('0.7519938324117397',) no score found for photo 1338640277 We are sending mail with results at report@fotonower.com args[1338640275] : ('0.7519938324117397',) no score found for photo 1338640275 We are sending mail with results at report@fotonower.com refus_total : 0.7519938324117397 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20699439 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 SELECT photo_id, url FROM MTRBack.photos ph WHERE photo_id IN (1338640177,1338690108,1338690104,1338690094,1338690091,1338640318,1338640277,1338640257,1338640256,1338640233,1338640231,1338640229,1338640184,1338640182,1338640179,1338640175,1338640275,1338640259,1338640187,1338690078) Found this number of photos: 20 begin to download photo : 1338640177 begin to download photo : 1338640318 begin to download photo : 1338640231 begin to download photo : 1338640175 download finish for photo 1338640177 begin to download photo : 1338690108 download finish for photo 1338640318 begin to download photo : 1338640277 download finish for photo 1338640175 begin to download photo : 1338640275 download finish for photo 1338640231 begin to download photo : 1338640229 download finish for photo 1338690108 begin to download photo : 1338690104 download finish for photo 1338640277 begin to download photo : 1338640257 download finish for photo 1338640275 begin to download photo : 1338640259 download finish for photo 1338690104 begin to download photo : 1338690094 download finish for photo 1338640229 begin to download photo : 1338640184 download finish for photo 1338690094 begin to download photo : 1338690091 download finish for photo 1338640257 begin to download photo : 1338640256 download finish for photo 1338640259 begin to download photo : 1338640187 download finish for photo 1338640184 begin to download photo : 1338640182 download finish for photo 1338690091 download finish for photo 1338640256 begin to download photo : 1338640233 download finish for photo 1338640187 begin to download photo : 1338690078 download finish for photo 1338640182 begin to download photo : 1338640179 download finish for photo 1338640233 download finish for photo 1338690078 download finish for photo 1338640179 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699439_21-02-2025_00_19_28.pdf results_Auto_P20699439_21-02-2025_00_19_28.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699439_21-02-2025_00_19_28.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('3459','20699439','results_Auto_P20699439_21-02-2025_00_19_28.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699439_21-02-2025_00_19_28.pdf','pdf','','0.79','0.7519938324117397') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1338690108, 1338690104, 1338690094, 1338690091, 1338690088, 1338690085, 1338690078, 1338640318, 1338640277, 1338640275] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690108', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690104', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690094', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690091', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690088', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690085', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690078', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640318', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640277', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640275', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 10 time used for this insertion : 0.016411304473876953 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.833749532699585 time spend to save output : 0.016765356063842773 total time spend for step 10 : 6.850514888763428 step11:split_time_score Fri Feb 21 00:19:35 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('17', 22),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 20022025 20699439 Nombre de photos uploadées : 22 / 23040 (0%) 20022025 20699439 Nombre de photos taguées (types de déchets): 0 / 22 (0%) 20022025 20699439 Nombre de photos taguées (volume) : 0 / 22 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 1.430511474609375e-05 ?????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.00136566162109375 elapsed_time : insert_dashboard_record_day_entry 0.028586387634277344 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699402 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699403 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699404 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699405 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699406 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699407 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699408 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699409 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699410 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699411 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699412 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699413 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699414 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699415 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699416 order by id desc limit 1 Qualite : 0.3767832473441416 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699417_21-02-2025_00_19_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699417 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699417 AND mptpi.`type`=4207 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699418 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699419 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699420 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699421 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699422 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699423 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699424 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699425 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699426 order by id desc limit 1 Qualite : 0.7519938324117397 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P20699439_21-02-2025_00_19_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699439 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11449 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 11452 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 11452 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11453 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11453 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11454 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11478 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11478 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11456 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11455 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11455 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11458 send_mail_cod have less inputs used (3) than in the step definition (5) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 11449 doesn't seem to be define in the database( WARNING : type of input 2 of step 11452 doesn't seem to be define in the database( WARNING : output 1 of step 11449 have datatype=2 whereas input 1 of step 11453 have datatype=7 WARNING : type of output 2 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11454 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 11453 doesn't seem to be define in the database( WARNING : type of input 1 of step 11456 doesn't seem to be define in the database( WARNING : type of output 1 of step 11456 doesn't seem to be define in the database( WARNING : type of input 3 of step 11455 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11456 have datatype=10 whereas input 3 of step 11458 have datatype=6 WARNING : type of input 5 of step 11458 doesn't seem to be define in the database( WARNING : output 0 of step 11477 have datatype=11 whereas input 5 of step 11458 have datatype=None WARNING : output 0 of step 11456 have datatype=10 whereas input 0 of step 11477 have datatype=18 WARNING : type of input 2 of step 11478 doesn't seem to be define in the database( WARNING : output 1 of step 11454 have datatype=7 whereas input 2 of step 11478 have datatype=None WARNING : type of output 3 of step 11478 doesn't seem to be define in the database( WARNING : type of input 2 of step 11456 doesn't seem to be define in the database( WARNING : output 0 of step 11453 have datatype=1 whereas input 0 of step 11454 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20699439 AND mptpi.`type`=3726 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699440 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699441 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699442 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699443 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699444 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699449 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699470 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699471 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699472 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699473 order by id desc limit 1 find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20699474 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'20022025': {'nb_upload': 22, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1338690108, 1338690104, 1338690094, 1338690091, 1338690088, 1338690085, 1338690078, 1338640318, 1338640277, 1338640275] Looping around the photos to save general results len do output : 1 /20699439Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690108', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690104', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690094', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690091', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690088', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690085', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338690078', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640318', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640277', None, None, None, None, None, '2601855') ('3459', None, None, None, None, None, None, None, '2601855') ('3459', '20699439', '1338640275', None, None, None, None, None, '2601855') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 11 time used for this insertion : 0.017532825469970703 save_final save missing photos in datou_result : time spend for datou_step_exec : 29.016376495361328 time spend to save output : 0.01776289939880371 total time spend for step 11 : 29.034139394760132 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 136.27user 210.24system 7:48.71elapsed 73%CPU (0avgtext+0avgdata 4144736maxresident)k 2493552inputs+120304outputs (29191major+11718425minor)pagefaults 0swaps