python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1269157 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2620850'] with mtr_portfolio_ids : ['20893122'] and first list_photo_ids : [] new path : /proc/1269157/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.2634444236755371 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Feb 27 21:20:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 5304 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-27 21:20:31.430776: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-27 21:20:31.455106: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-27 21:20:31.457490: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fe1f8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-27 21:20:31.457545: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-27 21:20:31.461564: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-27 21:20:31.601312: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3d1a5450 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-27 21:20:31.601366: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-27 21:20:31.602524: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 21:20:31.602931: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 21:20:31.605748: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 21:20:31.608383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 21:20:31.608860: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 21:20:31.612168: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 21:20:31.613428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 21:20:31.618428: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 21:20:31.619816: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 21:20:31.619894: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 21:20:31.620534: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-27 21:20:31.620549: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-27 21:20:31.620558: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-27 21:20:31.621611: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4843 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-27 21:20:31.883250: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 21:20:31.883335: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 21:20:31.883356: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 21:20:31.883374: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 21:20:31.883391: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 21:20:31.883408: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 21:20:31.883425: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 21:20:31.883443: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 21:20:31.884625: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 21:20:31.885722: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-27 21:20:31.885762: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-27 21:20:31.885784: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 21:20:31.885803: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-27 21:20:31.885823: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-27 21:20:31.885842: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-27 21:20:31.885862: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-27 21:20:31.885881: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-27 21:20:31.887076: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-27 21:20:31.887110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-27 21:20:31.887120: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-27 21:20:31.887129: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-27 21:20:31.888224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4843 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-27 21:20:39.985011: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-27 21:20:40.195020: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.71172 max: 147.43594 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 0.00013780593872070312 nb_pixel_total : 1019 time to create 1 rle with old method : 0.0021164417266845703 length of segment : 41 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 850 time to create 1 rle with old method : 0.0015642642974853516 length of segment : 43 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.52813 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 7.557868957519531e-05 nb_pixel_total : 1786 time to create 1 rle with old method : 0.0032155513763427734 length of segment : 42 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 1215 time to create 1 rle with old method : 0.00209808349609375 length of segment : 65 time for calcul the mask position with numpy : 6.556510925292969e-05 nb_pixel_total : 1589 time to create 1 rle with old method : 0.002925872802734375 length of segment : 34 Processing 1 images image shape: (400, 400, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -75.21563 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 6.246566772460938e-05 nb_pixel_total : 1093 time to create 1 rle with old method : 0.0020380020141601562 length of segment : 35 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.48125 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 233 time to create 1 rle with old method : 0.00035262107849121094 length of segment : 15 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 1874 time to create 1 rle with old method : 0.002342700958251953 length of segment : 67 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 486 time to create 1 rle with old method : 0.0006697177886962891 length of segment : 57 time for calcul the mask position with numpy : 7.915496826171875e-05 nb_pixel_total : 2096 time to create 1 rle with old method : 0.002628326416015625 length of segment : 108 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 554 time to create 1 rle with old method : 0.0007698535919189453 length of segment : 52 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 1972 time to create 1 rle with old method : 0.002498626708984375 length of segment : 114 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.01797 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 6.175041198730469e-05 nb_pixel_total : 460 time to create 1 rle with old method : 0.0009291172027587891 length of segment : 18 time for calcul the mask position with numpy : 9.083747863769531e-05 nb_pixel_total : 1801 time to create 1 rle with old method : 0.0032372474670410156 length of segment : 169 time for calcul the mask position with numpy : 0.0007829666137695312 nb_pixel_total : 51006 time to create 1 rle with old method : 0.07503747940063477 length of segment : 391 Processing 1 images image shape: (400, 400, 3) min: 35.00000 max: 240.00000 molded_images shape: (1, 640, 640, 3) min: -80.12969 max: 122.10391 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 32.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -76.82891 max: 148.71719 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 0.0001461505889892578 nb_pixel_total : 9088 time to create 1 rle with old method : 0.011037826538085938 length of segment : 113 time for calcul the mask position with numpy : 7.224082946777344e-05 nb_pixel_total : 2640 time to create 1 rle with old method : 0.0034618377685546875 length of segment : 47 Processing 1 images image shape: (400, 400, 3) min: 54.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -59.86797 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 961 time to create 1 rle with old method : 0.0017173290252685547 length of segment : 46 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 97 time to create 1 rle with old method : 0.0002753734588623047 length of segment : 20 time for calcul the mask position with numpy : 0.00022530555725097656 nb_pixel_total : 6445 time to create 1 rle with old method : 0.012900114059448242 length of segment : 176 time for calcul the mask position with numpy : 0.00030541419982910156 nb_pixel_total : 13154 time to create 1 rle with old method : 0.015761137008666992 length of segment : 165 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 478 time to create 1 rle with old method : 0.0007381439208984375 length of segment : 30 time for calcul the mask position with numpy : 4.029273986816406e-05 nb_pixel_total : 612 time to create 1 rle with old method : 0.0010633468627929688 length of segment : 10 time for calcul the mask position with numpy : 0.00010371208190917969 nb_pixel_total : 4001 time to create 1 rle with old method : 0.005057573318481445 length of segment : 137 Processing 1 images image shape: (400, 400, 3) min: 2.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -102.71953 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.00019693374633789062 nb_pixel_total : 3575 time to create 1 rle with old method : 0.0069653987884521484 length of segment : 51 time for calcul the mask position with numpy : 0.00016880035400390625 nb_pixel_total : 1894 time to create 1 rle with old method : 0.0035648345947265625 length of segment : 130 time for calcul the mask position with numpy : 0.0002105236053466797 nb_pixel_total : 2427 time to create 1 rle with old method : 0.0035343170166015625 length of segment : 61 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 193 time to create 1 rle with old method : 0.0003113746643066406 length of segment : 19 time for calcul the mask position with numpy : 0.0003447532653808594 nb_pixel_total : 10168 time to create 1 rle with old method : 0.012152433395385742 length of segment : 268 time for calcul the mask position with numpy : 0.00016689300537109375 nb_pixel_total : 1505 time to create 1 rle with old method : 0.002454042434692383 length of segment : 105 time for calcul the mask position with numpy : 4.1961669921875e-05 nb_pixel_total : 399 time to create 1 rle with old method : 0.0005671977996826172 length of segment : 19 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 8 time for calcul the mask position with numpy : 3.6716461181640625e-05 nb_pixel_total : 286 time to create 1 rle with old method : 0.00039076805114746094 length of segment : 30 time for calcul the mask position with numpy : 0.00019621849060058594 nb_pixel_total : 4135 time to create 1 rle with old method : 0.005354166030883789 length of segment : 224 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 1678 time to create 1 rle with old method : 0.0020682811737060547 length of segment : 85 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 189 time to create 1 rle with old method : 0.0002815723419189453 length of segment : 17 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.0002734661102294922 length of segment : 20 time for calcul the mask position with numpy : 7.653236389160156e-05 nb_pixel_total : 1285 time to create 1 rle with old method : 0.0017635822296142578 length of segment : 116 time for calcul the mask position with numpy : 0.0001666545867919922 nb_pixel_total : 2983 time to create 1 rle with old method : 0.003778219223022461 length of segment : 165 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 598 time to create 1 rle with old method : 0.0008215904235839844 length of segment : 29 Processing 1 images image shape: (280, 400, 3) min: 20.00000 max: 238.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 114.12734 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 558 time to create 1 rle with old method : 0.0007724761962890625 length of segment : 51 time for calcul the mask position with numpy : 0.0012819766998291016 nb_pixel_total : 105711 time to create 1 rle with old method : 0.15162086486816406 length of segment : 283 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.56484 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 8.082389831542969e-05 nb_pixel_total : 3679 time to create 1 rle with old method : 0.004628181457519531 length of segment : 68 time for calcul the mask position with numpy : 7.009506225585938e-05 nb_pixel_total : 2887 time to create 1 rle with old method : 0.003937482833862305 length of segment : 50 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 240 time to create 1 rle with old method : 0.00033354759216308594 length of segment : 18 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 693 time to create 1 rle with old method : 0.001008749008178711 length of segment : 38 time for calcul the mask position with numpy : 5.6743621826171875e-05 nb_pixel_total : 1014 time to create 1 rle with old method : 0.0015277862548828125 length of segment : 48 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 288 time to create 1 rle with old method : 0.000408172607421875 length of segment : 24 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 20 time to create 1 rle with old method : 7.748603820800781e-05 length of segment : 2 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 238 time to create 1 rle with old method : 0.00035071372985839844 length of segment : 29 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 1593 time to create 1 rle with old method : 0.0021593570709228516 length of segment : 46 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 969 time to create 1 rle with old method : 0.001337289810180664 length of segment : 32 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 302 time to create 1 rle with old method : 0.00042748451232910156 length of segment : 23 Processing 1 images image shape: (280, 400, 3) min: 35.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.77969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 18 time for calcul the mask position with numpy : 5.245208740234375e-05 nb_pixel_total : 815 time to create 1 rle with old method : 0.0010623931884765625 length of segment : 54 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 505 time to create 1 rle with old method : 0.0006997585296630859 length of segment : 38 time for calcul the mask position with numpy : 3.457069396972656e-05 nb_pixel_total : 394 time to create 1 rle with old method : 0.0005817413330078125 length of segment : 19 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 186 time to create 1 rle with old method : 0.00026917457580566406 length of segment : 18 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 596 time to create 1 rle with old method : 0.0008075237274169922 length of segment : 51 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 391 time to create 1 rle with old method : 0.0006246566772460938 length of segment : 47 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 237 time to create 1 rle with old method : 0.00035262107849121094 length of segment : 15 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 507 time to create 1 rle with old method : 0.0007159709930419922 length of segment : 32 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 345 time to create 1 rle with old method : 0.0005133152008056641 length of segment : 16 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 309 time to create 1 rle with old method : 0.0004477500915527344 length of segment : 22 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 164 time to create 1 rle with old method : 0.00025200843811035156 length of segment : 13 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 167 time to create 1 rle with old method : 0.00025343894958496094 length of segment : 17 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 84 time to create 1 rle with old method : 0.0001537799835205078 length of segment : 9 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 243 time to create 1 rle with old method : 0.0003705024719238281 length of segment : 15 time for calcul the mask position with numpy : 7.081031799316406e-05 nb_pixel_total : 2631 time to create 1 rle with old method : 0.0034270286560058594 length of segment : 74 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 201 time to create 1 rle with old method : 0.0003066062927246094 length of segment : 16 time for calcul the mask position with numpy : 3.933906555175781e-05 nb_pixel_total : 565 time to create 1 rle with old method : 0.0008158683776855469 length of segment : 34 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 540 time to create 1 rle with old method : 0.0007388591766357422 length of segment : 29 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.91250 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 147 time to create 1 rle with old method : 0.0002446174621582031 length of segment : 10 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 2166 time to create 1 rle with old method : 0.0029997825622558594 length of segment : 36 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 171 time to create 1 rle with old method : 0.0002560615539550781 length of segment : 25 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 454 time to create 1 rle with old method : 0.0006442070007324219 length of segment : 29 time for calcul the mask position with numpy : 7.486343383789062e-05 nb_pixel_total : 617 time to create 1 rle with old method : 0.0009295940399169922 length of segment : 56 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 131.72500 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.2928924560546875e-05 nb_pixel_total : 82 time to create 1 rle with old method : 0.00022602081298828125 length of segment : 34 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.45391 max: 145.54922 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 1253 time to create 1 rle with old method : 0.0015664100646972656 length of segment : 35 time for calcul the mask position with numpy : 4.00543212890625e-05 nb_pixel_total : 702 time to create 1 rle with old method : 0.0010161399841308594 length of segment : 53 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -113.38750 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 7.414817810058594e-05 nb_pixel_total : 1678 time to create 1 rle with old method : 0.002550363540649414 length of segment : 43 Processing 1 images image shape: (400, 400, 3) min: 34.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -70.70781 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 660 time to create 1 rle with old method : 0.0009109973907470703 length of segment : 28 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 865 time to create 1 rle with old method : 0.001169443130493164 length of segment : 31 time for calcul the mask position with numpy : 5.698204040527344e-05 nb_pixel_total : 577 time to create 1 rle with old method : 0.0009176731109619141 length of segment : 55 time for calcul the mask position with numpy : 5.53131103515625e-05 nb_pixel_total : 1369 time to create 1 rle with old method : 0.0019135475158691406 length of segment : 38 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -121.27422 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 212 time to create 1 rle with old method : 0.00035381317138671875 length of segment : 15 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 779 time to create 1 rle with old method : 0.001001119613647461 length of segment : 72 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 484 time to create 1 rle with old method : 0.0006663799285888672 length of segment : 43 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 440 time to create 1 rle with old method : 0.0006396770477294922 length of segment : 58 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 2025 time to create 1 rle with old method : 0.0024576187133789062 length of segment : 66 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.08828 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 5.14984130859375e-05 nb_pixel_total : 492 time to create 1 rle with old method : 0.001027822494506836 length of segment : 21 time for calcul the mask position with numpy : 7.843971252441406e-05 nb_pixel_total : 1118 time to create 1 rle with old method : 0.003456592559814453 length of segment : 59 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 107 time to create 1 rle with old method : 0.0002694129943847656 length of segment : 9 time for calcul the mask position with numpy : 0.0007917881011962891 nb_pixel_total : 52326 time to create 1 rle with old method : 0.08045077323913574 length of segment : 337 Processing 1 images image shape: (400, 400, 3) min: 27.00000 max: 240.00000 molded_images shape: (1, 640, 640, 3) min: -80.12969 max: 118.82266 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 43.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -69.56719 max: 148.40859 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.000186920166015625 nb_pixel_total : 10065 time to create 1 rle with old method : 0.011729955673217773 length of segment : 133 Processing 1 images image shape: (400, 400, 3) min: 59.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -55.80547 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 274 time to create 1 rle with old method : 0.0004398822784423828 length of segment : 41 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 784 time to create 1 rle with old method : 0.0010466575622558594 length of segment : 43 time for calcul the mask position with numpy : 4.172325134277344e-05 nb_pixel_total : 156 time to create 1 rle with old method : 0.0003800392150878906 length of segment : 16 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 1410 time to create 1 rle with old method : 0.0027709007263183594 length of segment : 19 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -105.12188 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 0.00011134147644042969 nb_pixel_total : 5017 time to create 1 rle with old method : 0.006194591522216797 length of segment : 72 time for calcul the mask position with numpy : 8.7738037109375e-05 nb_pixel_total : 1924 time to create 1 rle with old method : 0.0024602413177490234 length of segment : 123 time for calcul the mask position with numpy : 0.0003914833068847656 nb_pixel_total : 29030 time to create 1 rle with old method : 0.03510165214538574 length of segment : 273 time for calcul the mask position with numpy : 0.0001704692840576172 nb_pixel_total : 358 time to create 1 rle with old method : 0.00048661231994628906 length of segment : 21 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 790 time to create 1 rle with old method : 0.0011653900146484375 length of segment : 22 time for calcul the mask position with numpy : 0.00026607513427734375 nb_pixel_total : 8536 time to create 1 rle with old method : 0.01042795181274414 length of segment : 195 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 4.267692565917969e-05 nb_pixel_total : 301 time to create 1 rle with old method : 0.00041556358337402344 length of segment : 31 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 1689 time to create 1 rle with old method : 0.002141237258911133 length of segment : 85 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 190 time to create 1 rle with old method : 0.00027751922607421875 length of segment : 20 time for calcul the mask position with numpy : 0.0001933574676513672 nb_pixel_total : 4305 time to create 1 rle with old method : 0.005388498306274414 length of segment : 221 time for calcul the mask position with numpy : 3.647804260253906e-05 nb_pixel_total : 270 time to create 1 rle with old method : 0.00038361549377441406 length of segment : 27 time for calcul the mask position with numpy : 0.0001423358917236328 nb_pixel_total : 2933 time to create 1 rle with old method : 0.0038406848907470703 length of segment : 172 time for calcul the mask position with numpy : 0.00015211105346679688 nb_pixel_total : 4330 time to create 1 rle with old method : 0.005464792251586914 length of segment : 220 Processing 1 images image shape: (280, 400, 3) min: 19.00000 max: 249.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 115.75625 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 2 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 493 time to create 1 rle with old method : 0.0006647109985351562 length of segment : 51 time for calcul the mask position with numpy : 0.0009832382202148438 nb_pixel_total : 105805 time to create 1 rle with old method : 0.11986708641052246 length of segment : 282 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 10 time for calcul the mask position with numpy : 0.00010156631469726562 nb_pixel_total : 3690 time to create 1 rle with old method : 0.004685640335083008 length of segment : 72 time for calcul the mask position with numpy : 7.653236389160156e-05 nb_pixel_total : 3457 time to create 1 rle with old method : 0.0045318603515625 length of segment : 58 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 285 time to create 1 rle with old method : 0.00042748451232910156 length of segment : 25 time for calcul the mask position with numpy : 5.9604644775390625e-05 nb_pixel_total : 966 time to create 1 rle with old method : 0.0015079975128173828 length of segment : 48 time for calcul the mask position with numpy : 6.842613220214844e-05 nb_pixel_total : 2656 time to create 1 rle with old method : 0.003371000289916992 length of segment : 63 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 615 time to create 1 rle with old method : 0.0008738040924072266 length of segment : 34 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 1162 time to create 1 rle with old method : 0.0016486644744873047 length of segment : 42 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 265 time to create 1 rle with old method : 0.0003781318664550781 length of segment : 20 time for calcul the mask position with numpy : 2.8133392333984375e-05 nb_pixel_total : 20 time to create 1 rle with old method : 7.176399230957031e-05 length of segment : 3 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 895 time to create 1 rle with old method : 0.0013003349304199219 length of segment : 30 Processing 1 images image shape: (280, 400, 3) min: 37.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 149.20547 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 12 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 195 time to create 1 rle with old method : 0.00028967857360839844 length of segment : 19 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 531 time to create 1 rle with old method : 0.0007195472717285156 length of segment : 33 time for calcul the mask position with numpy : 3.910064697265625e-05 nb_pixel_total : 774 time to create 1 rle with old method : 0.0009353160858154297 length of segment : 54 time for calcul the mask position with numpy : 3.0279159545898438e-05 nb_pixel_total : 138 time to create 1 rle with old method : 0.00022530555725097656 length of segment : 12 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 513 time to create 1 rle with old method : 0.0007221698760986328 length of segment : 29 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 282 time to create 1 rle with old method : 0.00038623809814453125 length of segment : 16 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 530 time to create 1 rle with old method : 0.0007519721984863281 length of segment : 31 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 450 time to create 1 rle with old method : 0.0006198883056640625 length of segment : 23 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 371 time to create 1 rle with old method : 0.0005481243133544922 length of segment : 18 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 466 time to create 1 rle with old method : 0.0006318092346191406 length of segment : 24 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 101 time to create 1 rle with old method : 0.00017714500427246094 length of segment : 10 time for calcul the mask position with numpy : 3.409385681152344e-05 nb_pixel_total : 272 time to create 1 rle with old method : 0.0004208087921142578 length of segment : 21 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 145.06094 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 440 time to create 1 rle with old method : 0.0006132125854492188 length of segment : 25 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 136 time to create 1 rle with old method : 0.0002162456512451172 length of segment : 9 time for calcul the mask position with numpy : 2.9325485229492188e-05 nb_pixel_total : 175 time to create 1 rle with old method : 0.00024247169494628906 length of segment : 24 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 1401 time to create 1 rle with old method : 0.0020599365234375 length of segment : 47 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 476 time to create 1 rle with old method : 0.0006201267242431641 length of segment : 28 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 229 time to create 1 rle with old method : 0.0002987384796142578 length of segment : 24 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 134.03750 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.364418029785156e-05 nb_pixel_total : 94 time to create 1 rle with old method : 0.00025391578674316406 length of segment : 42 Detection mask done ! Trying to reset tf kernel 1269842 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 15 tf kernel not reseted sub process len(results) : 141 len(list_Values) 141 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 141 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 5304 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 141 chid ids of type : 4228 Number RLEs to save : 8669 save missing photos in datou_result : time spend for datou_step_exec : 22.74603581428528 time spend to save output : 0.7072055339813232 total time spend for step 1 : 23.4532413482666 step2:brightness Thu Feb 27 21:20:52 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for$œ’m…Îmo“LÒDüÜ;˜%gÏ?wêÁÅ·øîùovH0õÉa‡5£Ú*î Ø’ÃÌlÍ››S iyä”rÕO7ª“ž�%L]Ý×%±ºÇhk¶«·÷>v1­HB£®±ßÞÚd\�(e oIx¢>3´6BS%ÌØá“( œÛf$Ãhýé ¿¶åeÔŽôÚèHœ�‚`ݶ f{�Fo©Yò¿Ôó@00uMb’z-ëìXI$&ÂgfÖú¶7Ó´Þu|'K.ÌoP PÝÀùFË.Ðýoûò9B<~.’ïÅË[’´˜Ë<Ù­„$¯•¢·ä{1¹A�•.òbKxºL �¯�Ý·'¯u8n5 ’ºe,]ñH©–’ÆV¨ŒWwÃ$ùCƒel¹“|zys«™KŠi-ðqÊݬbk,wnGÿâ;¥ ~ ÖeÉrÍ’‰ÜÔ~' 1`V⦫¹-*[ÉñLÔKÄ'2@ŸÜþÐ仪 ²n‘Íß2¸N߈ƶµG•¢ói/U¢µ'Eï@¦`H湘;J•�¼¼ÜŽÅ +Jén#»¼‚6Ú´—ĹG•ü¡NÒGð'—Z!öáí¸‰Wi�»NJ @óàšAûÜZ|ª[¨ï$q}i�Ò·µQbtTEC$œ’m…Îmo“LÒDüÜ;˜%gÏ?wêÁÅ·øîùovH0õÉa‡5£Ú*î Ø’ÃÌlÍ››S iyä”rÕO7ª“ž�%L]Ý×%±ºÇhk¶«·÷>v1­HB£®±ßÞÚd\�(e oIx¢>3´6BS%ÌØá“( œÛf$Ãhýé ¿¶åeÔŽôÚèHœ�‚`ݶ f{�Fo©Yò¿Ôó@00uMb’z-ëìXI$&ÂgfÖú¶7Ó´Þu|'K.ÌoP PÝÀùFË.Ðýoûò9B<~.’ïÅË[’´˜Ë<Ù­„$¯•¢·ä{1¹A�•.òbKxºL �¯�Ý·'¯u8n5 ’ºe,]ñH©–’ÆV¨ŒWwÃ$ùCƒel¹“|zys«™KŠi-ðqÊݬbk,wnGÿâ;¥ ~ ÖeÉrÍ’‰ÜÔ~' 1`V⦫¹-*[ÉñLÔKÄ'2@ŸÜþÐ仪 ²n‘Íß2¸N߈ƶµG•¢ói/U¢µ'Eï@¦`H湘;J•�¼¼ÜŽÅ +Jén#»¼‚6Ú´—ĹG•ü¡NÒGð'—Z!öáí¸‰Wi�»NJ @óàšAûÜZ|ª[¨ï$q}i�Ò·µQbtTEC$œ’m…Îmo“LÒDüÜ;˜%gÏ?wêÁÅ·øîùovH0õÉa‡5£Ú*î Ø’ÃÌlÍ››S iyä”rÕO7ª“ž�%L]Ý×%±ºÇhk¶«·÷>v1­HB£®±ßÞÚd\�(e oIx¢>3´6BS%ÌØá“( œÛf$Ãhýé ¿¶åeÔŽôÚèHœ�‚`ݶ f{�Fo©Yò¿Ôó@00uMb’z-ëìXI$&ÂgfÖú¶7Ó´Þu|'K.ÌoP PÝÀùFË.Ðýoûò9B<~.’ïÅË[’´˜Ë<Ù­„$¯•¢·ä{1¹A�•.òbKxºL �¯�Ý·'¯u8n5 ’ºe,]ñH©–’ÆV¨ŒWwÃ$ùCƒel¹“|zys«™KŠi-ðqÊݬbk,wnGÿâ;¥ ~ ÖeÉrÍ’‰ÜÔ~' 1`V⦫¹-*[ÉñLÔKÄ'2@ŸÜþÐ仪 ²n‘Íß2¸N߈ƶµG•¢ói/U¢µ'Eï@¦`H湘;J•�¼¼ÜŽÅ +Jén#»¼‚6Ú´—ĹG•ü¡NÒGð'—Z!öáí¸‰Wi�»NJ @óàšAûÜZ|ª[¨ï$q}i�Ò·µQbtTEC$œ’m…Îmo“LÒDüÜ;˜%gÏ?wêÁÅ·øîùovH0õÉa‡5£Ú*î Ø’ÃÌlÍ››S iyä”rÕO7ª“ž�%L]Ý×%±ºÇhk¶«·÷>v1­HB£®±ßÞÚd\�(e oIx¢>3´6BS%ÌØá“( œÛf$Ãhýé ¿¶åeÔŽôÚèHœ�‚`ݶ f{�Fo©Yò¿Ôó@00uMb’z-ëìXI$&ÂgfÖú¶7Ó´Þu|'K.ÌoP PÝÀùFË.Ðýoûò9B<~.’ïÅË[’´˜Ë<Ù­„$¯•¢·ä{1¹A�•.òbKxºL �¯�Ý·'¯u8n5 ’ºe,]ñH©–’ÆV¨ŒWwÃ$ùCƒel¹“|zys«™KŠi-ðqÊݬbk,wnGÿâ;¥ ~ ÖeÉrÍ’‰ÜÔ~' 1`V⦫¹-*[ÉñLÔKÄ'2@ŸÜþÐ仪 ²n‘Íß2¸N߈ƶµG•¢ói/U¢µ'Eï@¦`H湘;J•�¼¼ÜŽÅ +Jén#»¼‚6Ú´—ĹG•ü¡NÒGð'—Z!öáí¸‰Wi�»NJ @óàšAûÜZ|ª[¨ï$q}i�Ò·µQbtTEC$œ’m…Îmo“LÒDüÜ;˜%gÏ?wêÁÅ·øîùovH0õÉa‡5£Ú*î Ø’ÃÌlÍ››S iyä”rÕO7ª“ž�%L]Ý×%±ºÇhk¶«·÷>v1­HB£®±ßÞÚd\�(e oIx¢>3´6BS%ÌØá“( œÛf$Ãhýé ¿¶åeÔŽôÚèHœ�‚`ݶ f{�Fo©Yò¿Ôó@00uMb’z-ëìXI$&ÂgfÖú¶7Ó´Þu|'K.ÌoP PÝÀùFË.Ðýoûò9B<~.’ïÅË[’´˜Ë<Ù­„$¯•¢·ä{1¹A�•.òbKxºL �¯�Ý·'¯u8n5 ’ºe,]ñH©–’ÆV¨ŒWwÃ$ùCƒel¹“|zys«™KŠi-ðqÊݬbk,wnGÿâ;¥ ~ ÖeÉrÍ’‰ÜÔ~' 1`V⦫¹-*[ÉñLÔKÄ'2@ŸÜþÐ仪 ²n‘Íß2¸N߈ƶµG•¢ói/U¢µ'Eï@¦`H湘;J•�¼¼ÜŽÅ +Jén#»¼‚6Ú´—ĹG•ü¡NÒGð'—Z!öáí¸‰Wi�»NJ @óàšAûÜZ|ª[¨ï$q}i�Ò·µQbtTEC$œ’m…Îmo“LÒDüÜ;˜%gÏ?wêÁÅ·øîùovH0õÉa‡5£Ú*î Ø’ÃÌlÍ››S iyä”rÕO7ª“ž�%L]Ý×%±ºÇhk¶«·÷>v1­HB£®±ßÞÚd\�(e oIx¢>3´6BS%ÌØá“( œÛf$Ãhýé ¿¶åeÔŽôÚèHœ�‚`ݶ f{�Fo©Yò¿Ôó@00uMb’z-ëìXI$&ÂgfÖú¶7Ó´Þu|'K.ÌoP PÝÀùFË.Ðýoûò9B<~.’ïÅË[’´˜Ë<Ù­„$¯•¢·ä{1¹A�•.òbKxºL �¯�Ý·'¯u8n5 ’ºe,]ñH©–’ÆV¨ŒWwÃ$ùCƒel¹“|zys«™KŠi-ðqÊݬbk,wnGÿâ;¥ ~ ÖeÉrÍ’‰ÜÔ~' 1`V⦫¹-*[ÉñLÔKÄ'2@ŸÜþÐ仪 ²n‘Íß2¸N߈ƶµG•¢ói/U¢µ'Eï@¦`H湘;J•�¼¼ÜŽÅ +Jén#»¼‚6Ú´—ĹG•ü¡NÒGð'—Z!öáí¸‰Wi�»NJ @óàšAûÜZ|ª[¨ï$q}i�Ò·µQbtTEC$œ’m…Îmo“LÒDüÜ;˜%gÏ?wêÁÅ·øîùovH0õÉa‡5£Ú*î Ø’ÃÌlÍ››S iyä”rÕO7ª“ž�%L]Ý×%±ºÇhk¶«·÷>v1­HB£®±ßÞÚd\�(e oIx¢>3´6BS%ÌØá“( œÛf$Ãhýé ¿¶åeÔŽôÚèHœ�‚`ݶ f{�Fo©Yò¿Ôó@00uMb’z-ëìXI$&ÂgfÖú¶7Ó´Þu|'K.ÌoP PÝÀùFË.Ðýoûò9B<~.’ïÅË[’´˜Ë<Ù­„$¯•¢·ä{1¹A�•.òbKxºL �¯�Ý·'¯u8n5 ’ºe,]ñH©–’ÆV¨ŒWwÃ$ùCƒel¹“|zys«™KŠi-ðqÊݬbk,wnGÿâ;¥ ~ ÖeÉrÍ’‰ÜÔ~' 1`V⦫¹-*[ÉñLÔKÄ'2@ŸÜþÐ仪 ²n‘Íß2¸N߈ƶµG•¢ói/U¢µ'Eï@¦`H湘;J•�¼¼ÜŽÅ +Jén#»¼‚6Ú´—ĹG•ü¡NÒGð'—Z!öáí¸‰Wi�»NJ @óàšAûÜZ|ª[¨ï$q}i�Ò·µQbtTEC$œ’m…Îmo“LÒDüÜ;˜%gÏ?wêÁÅ·øîùovH0õÉa‡5£Ú*î Ø’ÃÌlÍ››S iyä”rÕO7ª“ž�%L]Ý×%±ºÇhk¶«·÷>v1­HB£®±ßÞÚd\�(e oIx¢>3´6BS%ÌØá“( œÛf$Ãhýé ¿¶åeÔŽôÚèHœ�‚`ݶ f{�Fo©Yò¿Ôó@00uMb’z-ëìXI$&ÂgfÖú¶7Ó´Þu|'K.ÌoP PÝÀùFË.Ðýoûò9B<~.’ïÅË[’´˜Ë<Ù­„$¯•¢·ä{1¹A�•.òbKxºL �¯�Ý·'¯u8n5 ’ºe,]ñH©–’ÆV¨ŒWwÃ$ùCƒel¹“|zys«™KŠi-ðqÊݬbk,wnGÿâ;¥ ~ ÖeÉrÍ’‰ÜÔ~' 1`V⦫¹-*[ÉñLÔKÄ'2@ŸÜþÐ仪 ²n‘Íß2¸N߈ƶµG•¢ói/U¢µ'Eï@¦`H湘;J•�¼¼ÜŽÅ +Jén#»¼‚6Ú´—ĹG•ü¡NÒGð'—Z!öáí¸‰Wi�»NJ @óàšAûÜZ|ª[¨ï$q}i�Ò·µQbtTEC849296569824 time for calcul the mask position with numpy : 0.008713245391845703 nb_pixel_total : 1801 time to create 1 rle with old method : 0.0019693374633789062 time for calcul the mask position with numpy : 0.008831262588500977 nb_pixel_total : 51006 time to create 1 rle with old method : 0.05579209327697754 time for calcul the mask position with numpy : 0.008201122283935547 nb_pixel_total : 1019 time to create 1 rle with old method : 0.0011301040649414062 time for calcul the mask position with numpy : 0.008254051208496094 nb_pixel_total : 850 time to create 1 rle with old method : 0.0009837150573730469 time for calcul the mask position with numpy : 0.008309125900268555 nb_pixel_total : 1786 time to create 1 rle with old method : 0.0020253658294677734 time for calcul the mask position with numpy : 0.008292436599731445 nb_pixel_total : 420 time to create 1 rle with old method : 0.0004868507385253906 time for calcul the mask position with numpy : 0.008681774139404297 nb_pixel_total : 1093 time to create 1 rle with old method : 0.0011515617370605469 time for calcul the mask position with numpy : 0.008302450180053711 nb_pixel_total : 1215 time to create 1 rle with old method : 0.001352071762084961 time for calcul the mask position with numpy : 0.008376121520996094 nb_pixel_total : 1589 time to create 1 rle with old method : 0.0017986297607421875 time for calcul the mask position with numpy : 0.008456230163574219 nb_pixel_total : 233 time to create 1 rle with old method : 0.0002765655517578125 time for calcul the mask position with numpy : 0.008412599563598633 nb_pixel_total : 84 time to create 1 rle with old method : 0.0001373291015625 time for calcul the mask position with numpy : 0.008126974105834961 nb_pixel_total : 2096 time to create 1 rle with old method : 0.0023221969604492188 time for calcul the mask position with numpy : 0.008266448974609375 nb_pixel_total : 159 time to create 1 rle with old method : 0.00023603439331054688 time for calcul the mask position with numpy : 0.008259057998657227 nb_pixel_total : 1874 time to create 1 rle with old method : 0.002124309539794922 time for calcul the mask position with numpy : 0.008401632308959961 nb_pixel_total : 460 time to create 1 rle with old method : 0.0005528926849365234 time for calcul the mask position with numpy : 0.008467674255371094 nb_pixel_total : 612 time to create 1 rle with old method : 0.0007500648498535156 time for calcul the mask position with numpy : 0.008278369903564453 nb_pixel_total : 961 time to create 1 rle with old method : 0.0010955333709716797 time for calcul the mask position with numpy : 0.0082550048828125 nb_pixel_total : 6445 time to create 1 rle with old method : 0.00722503662109375 time for calcul the mask position with numpy : 0.008277654647827148 nb_pixel_total : 520 time to create 1 rle with old method : 0.0006954669952392578 time for calcul the mask position with numpy : 0.008337020874023438 nb_pixel_total : 819 time to create 1 rle with old method : 0.0009887218475341797 time for calcul the mask position with numpy : 0.008350372314453125 nb_pixel_total : 13154 time to create 1 rle with old method : 0.014525175094604492 time for calcul the mask position with numpy : 0.0083465576171875 nb_pixel_total : 97 time to create 1 rle with old method : 0.00012063980102539062 time for calcul the mask position with numpy : 0.010171175003051758 nb_pixel_total : 189 time to create 1 rle with old method : 0.0004525184631347656 time for calcul the mask position with numpy : 0.01200246810913086 nb_pixel_total : 3575 time to create 1 rle with old method : 0.003945589065551758 time for calcul the mask position with numpy : 0.008191585540771484 nb_pixel_total : 193 time to create 1 rle with old method : 0.0002262592315673828 time for calcul the mask position with numpy : 0.008182048797607422 nb_pixel_total : 9088 time to create 1 rle with old method : 0.009733438491821289 time for calcul the mask position with numpy : 0.007945537567138672 nb_pixel_total : 286 time to create 1 rle with old method : 0.00029969215393066406 time for calcul the mask position with numpy : 0.00770878791809082 nb_pixel_total : 1285 time to create 1 rle with old method : 0.0014333724975585938 time for calcul the mask position with numpy : 0.007631540298461914 nb_pixel_total : 4135 time to create 1 rle with old method : 0.0043108463287353516 time for calcul the mask position with numpy : 0.008019685745239258 nb_pixel_total : 399 time to create 1 rle with old method : 0.00047707557678222656 time for calcul the mask position with numpy : 0.007990837097167969 nb_pixel_total : 175 time to create 1 rle with old method : 0.00022840499877929688 time for calcul the mask position with numpy : 0.007601261138916016 nb_pixel_total : 1894 time to create 1 rle with old method : 0.002145528793334961 time for calcul the mask position with numpy : 0.007688999176025391 nb_pixel_total : 10168 time to create 1 rle with old method : 0.011006593704223633 time for calcul the mask position with numpy : 0.008004426956176758 nb_pixel_total : 230 time to create 1 rle with old method : 0.0002949237823486328 time for calcul the mask position with numpy : 0.008104324340820312 nb_pixel_total : 570 time to create 1 rle with old method : 0.0005962848663330078 time for calcul the mask position with numpy : 0.008017539978027344 nb_pixel_total : 2640 time to create 1 rle with old method : 0.0031397342681884766 time for calcul the mask position with numpy : 0.008150100708007812 nb_pixel_total : 1505 time to create 1 rle with old method : 0.0018799304962158203 time for calcul the mask position with numpy : 0.007992744445800781 nb_pixel_total : 478 time to create 1 rle with old method : 0.0005853176116943359 time for calcul the mask position with numpy : 0.008130550384521484 nb_pixel_total : 1678 time to create 1 rle with old method : 0.001880645751953125 time for calcul the mask position with numpy : 0.008416891098022461 nb_pixel_total : 105332 time to create 1 rle with old method : 0.11339092254638672 time for calcul the mask position with numpy : 0.00798344612121582 nb_pixel_total : 82 time to create 1 rle with old method : 0.0001277923583984375 time for calcul the mask position with numpy : 0.0083465576171875 nb_pixel_total : 20 time to create 1 rle with old method : 3.337860107421875e-05 time for calcul the mask position with numpy : 0.008202791213989258 nb_pixel_total : 288 time to create 1 rle with old method : 0.00033354759216308594 time for calcul the mask position with numpy : 0.008116722106933594 nb_pixel_total : 14 time to create 1 rle with old method : 3.457069396972656e-05 time for calcul the mask position with numpy : 0.007886648178100586 nb_pixel_total : 1593 time to create 1 rle with old method : 0.0017364025115966797 time for calcul the mask position with numpy : 0.008016824722290039 nb_pixel_total : 29 time to create 1 rle with old method : 4.553794860839844e-05 time for calcul the mask position with numpy : 0.007878541946411133 nb_pixel_total : 243 time to create 1 rle with old method : 0.00025653839111328125 time for calcul the mask position with numpy : 0.007911920547485352 nb_pixel_total : 164 time to create 1 rle with old method : 0.00017452239990234375 time for calcul the mask position with numpy : 0.007720232009887695 nb_pixel_total : 84 time to create 1 rle with old method : 0.00010561943054199219 time for calcul the mask position with numpy : 0.007875680923461914 nb_pixel_total : 345 time to create 1 rle with old method : 0.00035262107849121094 time for calcul the mask position with numpy : 0.00832676887512207 nb_pixel_total : 2887 time to create 1 rle with old method : 0.003214120864868164 time for calcul the mask position with numpy : 0.008368492126464844 nb_pixel_total : 237 time to create 1 rle with old method : 0.0002791881561279297 time for calcul the mask position with numpy : 0.008402109146118164 nb_pixel_total : 596 time to create 1 rle with old method : 0.0007302761077880859 time for calcul the mask position with numpy : 0.008375167846679688 nb_pixel_total : 240 time to create 1 rle with old method : 0.000278472900390625 time for calcul the mask position with numpy : 0.008361101150512695 nb_pixel_total : 309 time to create 1 rle with old method : 0.0003731250762939453 time for calcul the mask position with numpy : 0.008882284164428711 nb_pixel_total : 565 time to create 1 rle with old method : 0.001041412353515625 time for calcul the mask position with numpy : 0.00937199592590332 nb_pixel_total : 167 time to create 1 rle with old method : 0.0002028942108154297 time for calcul the mask position with numpy : 0.008369207382202148 nb_pixel_total : 3679 time to create 1 rle with old method : 0.004164218902587891 time for calcul the mask position with numpy : 0.008294343948364258 nb_pixel_total : 394 time to create 1 rle with old method : 0.00044989585876464844 time for calcul the mask position with numpy : 0.008322954177856445 nb_pixel_total : 238 time to create 1 rle with old method : 0.0002677440643310547 time for calcul the mask position with numpy : 0.00809931755065918 nb_pixel_total : 201 time to create 1 rle with old method : 0.0002346038818359375 time for calcul the mask position with numpy : 0.007911443710327148 nb_pixel_total : 100 time to create 1 rle with old method : 0.00014066696166992188 time for calcul the mask position with numpy : 0.00807499885559082 nb_pixel_total : 693 time to create 1 rle with old method : 0.0008170604705810547 time for calcul the mask position with numpy : 0.008322000503540039 nb_pixel_total : 2166 time to create 1 rle with old method : 0.002519369125366211 time for calcul the mask position with numpy : 0.009213685989379883 nb_pixel_total : 171 time to create 1 rle with old method : 0.00020384788513183594 time for calcul the mask position with numpy : 0.008201837539672852 nb_pixel_total : 39 time to create 1 rle with old method : 7.510185241699219e-05 time for calcul the mask position with numpy : 0.008092164993286133 nb_pixel_total : 507 time to create 1 rle with old method : 0.0005266666412353516 time for calcul the mask position with numpy : 0.008067131042480469 nb_pixel_total : 505 time to create 1 rle with old method : 0.0005731582641601562 time for calcul the mask position with numpy : 0.007940292358398438 nb_pixel_total : 2301 time to create 1 rle with old method : 0.002386808395385742 time for calcul the mask position with numpy : 0.008055686950683594 nb_pixel_total : 617 time to create 1 rle with old method : 0.0006642341613769531 time for calcul the mask position with numpy : 0.00774073600769043 nb_pixel_total : 558 time to create 1 rle with old method : 0.0006673336029052734 time for calcul the mask position with numpy : 0.007840871810913086 nb_pixel_total : 1014 time to create 1 rle with old method : 0.0011501312255859375 time for calcul the mask position with numpy : 0.008071422576904297 nb_pixel_total : 815 time to create 1 rle with old method : 0.0009322166442871094 time for calcul the mask position with numpy : 0.008087873458862305 nb_pixel_total : 391 time to create 1 rle with old method : 0.00048661231994628906 time for calcul the mask position with numpy : 0.008176565170288086 nb_pixel_total : 186 time to create 1 rle with old method : 0.0002224445343017578 time for calcul the mask position with numpy : 0.008691072463989258 nb_pixel_total : 147 time to create 1 rle with old method : 0.0001747608184814453 create new chi : 1.2923831939697266 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.015288829803466797 batch 1 Loaded 77 chid ids of type : 4230 Number RLEs to save : 9694 TO DO : save crop sub photo not yet done ! save time : 0.5875537395477295 nb_obj : 65 nb_hashtags : 6 time to prepare the origin masks : 1.7788498401641846 time for calcul the mask position with numpy : 0.036415815353393555 nb_pixel_total : 1816567 time to create 1 rle with new method : 0.05823493003845215 time for calcul the mask position with numpy : 0.006400585174560547 nb_pixel_total : 52309 time to create 1 rle with old method : 0.0589146614074707 time for calcul the mask position with numpy : 0.005963325500488281 nb_pixel_total : 1253 time to create 1 rle with old method : 0.0014317035675048828 time for calcul the mask position with numpy : 0.006136894226074219 nb_pixel_total : 702 time to create 1 rle with old method : 0.0008308887481689453 time for calcul the mask position with numpy : 0.005840778350830078 nb_pixel_total : 1678 time to create 1 rle with old method : 0.0019109249114990234 time for calcul the mask position with numpy : 0.0059621334075927734 nb_pixel_total : 660 time to create 1 rle with old method : 0.0007452964782714844 time for calcul the mask position with numpy : 0.005880117416381836 nb_pixel_total : 638 time to create 1 rle with old method : 0.0007646083831787109 time for calcul the mask position with numpy : 0.005948543548583984 nb_pixel_total : 440 time to create 1 rle with old method : 0.0005083084106445312 time for calcul the mask position with numpy : 0.005989789962768555 nb_pixel_total : 865 time to create 1 rle with old method : 0.0009553432464599609 time for calcul the mask position with numpy : 0.005982398986816406 nb_pixel_total : 1369 time to create 1 rle with old method : 0.001561880111694336 time for calcul the mask position with numpy : 0.006163835525512695 nb_pixel_total : 212 time to create 1 rle with old method : 0.00026154518127441406 time for calcul the mask position with numpy : 0.006060123443603516 nb_pixel_total : 1118 time to create 1 rle with old method : 0.00135040283203125 time for calcul the mask position with numpy : 0.006074428558349609 nb_pixel_total : 484 time to create 1 rle with old method : 0.0005841255187988281 time for calcul the mask position with numpy : 0.006116390228271484 nb_pixel_total : 577 time to create 1 rle with old method : 0.0008406639099121094 time for calcul the mask position with numpy : 0.006136894226074219 nb_pixel_total : 107 time to create 1 rle with old method : 0.00014519691467285156 time for calcul the mask position with numpy : 0.0061817169189453125 nb_pixel_total : 2025 time to create 1 rle with old method : 0.0034558773040771484 time for calcul the mask position with numpy : 0.007322072982788086 nb_pixel_total : 492 time to create 1 rle with old method : 0.0009033679962158203 time for calcul the mask position with numpy : 0.008565902709960938 nb_pixel_total : 1410 time to create 1 rle with old method : 0.002466917037963867 time for calcul the mask position with numpy : 0.011479854583740234 nb_pixel_total : 784 time to create 1 rle with old method : 0.0009229183197021484 time for calcul the mask position with numpy : 0.010231494903564453 nb_pixel_total : 156 time to create 1 rle with old method : 0.0002434253692626953 time for calcul the mask position with numpy : 0.010564088821411133 nb_pixel_total : 358 time to create 1 rle with old method : 0.00044798851013183594 time for calcul the mask position with numpy : 0.010174751281738281 nb_pixel_total : 274 time to create 1 rle with old method : 0.0003376007080078125 time for calcul the mask position with numpy : 0.009906768798828125 nb_pixel_total : 190 time to create 1 rle with old method : 0.00022983551025390625 time for calcul the mask position with numpy : 0.010441780090332031 nb_pixel_total : 5017 time to create 1 rle with old method : 0.005859375 time for calcul the mask position with numpy : 0.009930610656738281 nb_pixel_total : 10065 time to create 1 rle with old method : 0.011829376220703125 time for calcul the mask position with numpy : 0.010480165481567383 nb_pixel_total : 270 time to create 1 rle with old method : 0.0003299713134765625 time for calcul the mask position with numpy : 0.009918451309204102 nb_pixel_total : 301 time to create 1 rle with old method : 0.00036215782165527344 time for calcul the mask position with numpy : 0.010012149810791016 nb_pixel_total : 29030 time to create 1 rle with old method : 0.03171181678771973 time for calcul the mask position with numpy : 0.009618997573852539 nb_pixel_total : 4305 time to create 1 rle with old method : 0.004848003387451172 time for calcul the mask position with numpy : 0.009955406188964844 nb_pixel_total : 8508 time to create 1 rle with old method : 0.009417057037353516 time for calcul the mask position with numpy : 0.010240793228149414 nb_pixel_total : 179 time to create 1 rle with old method : 0.0003077983856201172 time for calcul the mask position with numpy : 0.009990930557250977 nb_pixel_total : 133 time to create 1 rle with old method : 0.00021696090698242188 time for calcul the mask position with numpy : 0.0095977783203125 nb_pixel_total : 1924 time to create 1 rle with old method : 0.0020890235900878906 time for calcul the mask position with numpy : 0.006090641021728516 nb_pixel_total : 1689 time to create 1 rle with old method : 0.002568483352661133 time for calcul the mask position with numpy : 0.00640106201171875 nb_pixel_total : 790 time to create 1 rle with old method : 0.0009298324584960938 time for calcul the mask position with numpy : 0.006978273391723633 nb_pixel_total : 105400 time to create 1 rle with old method : 0.12272357940673828 time for calcul the mask position with numpy : 0.012823820114135742 nb_pixel_total : 94 time to create 1 rle with old method : 0.00018858909606933594 time for calcul the mask position with numpy : 0.011014699935913086 nb_pixel_total : 20 time to create 1 rle with old method : 4.839897155761719e-05 time for calcul the mask position with numpy : 0.009680986404418945 nb_pixel_total : 883 time to create 1 rle with old method : 0.00104522705078125 time for calcul the mask position with numpy : 0.009984254837036133 nb_pixel_total : 2656 time to create 1 rle with old method : 0.002981901168823242 time for calcul the mask position with numpy : 0.009884119033813477 nb_pixel_total : 138 time to create 1 rle with old method : 0.0001838207244873047 time for calcul the mask position with numpy : 0.009593725204467773 nb_pixel_total : 101 time to create 1 rle with old method : 0.00014472007751464844 time for calcul the mask position with numpy : 0.00984644889831543 nb_pixel_total : 371 time to create 1 rle with old method : 0.0004723072052001953 time for calcul the mask position with numpy : 0.00982809066772461 nb_pixel_total : 3457 time to create 1 rle with old method : 0.003910541534423828 time for calcul the mask position with numpy : 0.00981903076171875 nb_pixel_total : 282 time to create 1 rle with old method : 0.0003447532653808594 time for calcul the mask position with numpy : 0.009842157363891602 nb_pixel_total : 265 time to create 1 rle with old method : 0.0003223419189453125 time for calcul the mask position with numpy : 0.009868621826171875 nb_pixel_total : 272 time to create 1 rle with old method : 0.00033283233642578125 time for calcul the mask position with numpy : 0.010533571243286133 nb_pixel_total : 1157 time to create 1 rle with old method : 0.0013785362243652344 time for calcul the mask position with numpy : 0.006520986557006836 nb_pixel_total : 3690 time to create 1 rle with old method : 0.004064798355102539 time for calcul the mask position with numpy : 0.006072521209716797 nb_pixel_total : 18 time to create 1 rle with old method : 4.649162292480469e-05 time for calcul the mask position with numpy : 0.006027936935424805 nb_pixel_total : 450 time to create 1 rle with old method : 0.0004990100860595703 time for calcul the mask position with numpy : 0.006096839904785156 nb_pixel_total : 285 time to create 1 rle with old method : 0.00034117698669433594 time for calcul the mask position with numpy : 0.0059320926666259766 nb_pixel_total : 440 time to create 1 rle with old method : 0.0005011558532714844 time for calcul the mask position with numpy : 0.006018161773681641 nb_pixel_total : 62 time to create 1 rle with old method : 0.00010180473327636719 time for calcul the mask position with numpy : 0.006052970886230469 nb_pixel_total : 175 time to create 1 rle with old method : 0.0002205371856689453 time for calcul the mask position with numpy : 0.0062160491943359375 nb_pixel_total : 615 time to create 1 rle with old method : 0.0007176399230957031 time for calcul the mask position with numpy : 0.006138324737548828 nb_pixel_total : 1276 time to create 1 rle with old method : 0.001491546630859375 time for calcul the mask position with numpy : 0.006061553955078125 nb_pixel_total : 513 time to create 1 rle with old method : 0.0005981922149658203 time for calcul the mask position with numpy : 0.006613254547119141 nb_pixel_total : 476 time to create 1 rle with old method : 0.0005626678466796875 time for calcul the mask position with numpy : 0.006088733673095703 nb_pixel_total : 530 time to create 1 rle with old method : 0.0006155967712402344 time for calcul the mask position with numpy : 0.006379842758178711 nb_pixel_total : 493 time to create 1 rle with old method : 0.0006220340728759766 time for calcul the mask position with numpy : 0.006260395050048828 nb_pixel_total : 966 time to create 1 rle with old method : 0.0011413097381591797 time for calcul the mask position with numpy : 0.0064373016357421875 nb_pixel_total : 774 time to create 1 rle with old method : 0.0008738040924072266 time for calcul the mask position with numpy : 0.0063304901123046875 nb_pixel_total : 531 time to create 1 rle with old method : 0.0006680488586425781 time for calcul the mask position with numpy : 0.006050586700439453 nb_pixel_total : 195 time to create 1 rle with old method : 0.00024580955505371094 time for calcul the mask position with numpy : 0.0076220035552978516 nb_pixel_total : 136 time to create 1 rle with old method : 0.0002906322479248047 create new chi : 0.9093954563140869 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.002020597457885742 batch 1 Loaded 66 chid ids of type : 4230 Number RLEs to save : 8145 TO DO : save crop sub photo not yet done ! save time : 0.5292239189147949 map_output_result : {1339978692: (0.0, 'Should be the crop_list due to order', 0.0), 1339978642: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1339978692, 1339978642] Looping around the photos to save general results len do output : 2 /1339978692.Didn't retrieve data . /1339978642.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2620850') ('4234', '20893122', '1339978692', None, None, None, None, None, '2620850') ('4234', None, None, None, None, None, None, None, '2620850') ('4234', '20893122', '1339978642', None, None, None, None, None, '2620850') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.015018939971923828 save_final save missing photos in datou_result : time spend for datou_step_exec : 6.575717210769653 time spend to save output : 0.01526331901550293 total time spend for step 4 : 6.590980529785156 step5:crop_condition Thu Feb 27 21:21:01 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 143 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 40 About to insert : list_path_to_insert length 40 new photo from crops ! About to upload 40 photos upload in portfolio : 4869462 init cache_photo without model_param we have 40 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740687662_1269157 we have uploaded 40 photos in the portfolio 4869462 time of upload the photos Elapsed time : 10.106890678405762 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 20 About to insert : list_path_to_insert length 20 new photo from crops ! About to upload 20 photos upload in portfolio : 4869462 init cache_photo without model_param we have 20 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740687673_1269157 we have uploaded 20 photos in the portfolio 4869462 time of upload the photos Elapsed time : 4.9622015953063965 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740687678_1269157 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.8497531414031982 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 4 About to insert : list_path_to_insert length 4 new photo from crops ! About to upload 4 photos upload in portfolio : 4869462 init cache_photo without model_param we have 4 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740687679_1269157 we have uploaded 4 photos in the portfolio 4869462 time of upload the photos Elapsed time : 2.178799867630005 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 24 About to insert : list_path_to_insert length 24 new photo from crops ! About to upload 24 photos upload in portfolio : 4869462 init cache_photo without model_param we have 24 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740687682_1269157 we have uploaded 24 photos in the portfolio 4869462 time of upload the photos Elapsed time : 6.605534791946411 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1339978692, 1339978642] Looping around the photos to save general results len do output : 90 /1339985198Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985199Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985200Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985201Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985202Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985203Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985204Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985205Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985206Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985207Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985208Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985209Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985210Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985211Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985212Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985213Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985214Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985215Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985216Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985217Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985218Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985219Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985221Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985222Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985223Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985224Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985225Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985226Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985227Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985228Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985229Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985230Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985231Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985232Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985233Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985234Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985235Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985236Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985237Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985238Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985251Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985252Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985253Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985254Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985255Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985256Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985257Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985258Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985259Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985260Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985261Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985262Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985263Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985264Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985265Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985266Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985267Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985268Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985269Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985270Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985272Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985273Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985276Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985277Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985278Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985279Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985284Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985285Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985286Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985287Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985288Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985289Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985290Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985291Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985292Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985293Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985294Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985295Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985296Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985297Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985298Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985299Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985300Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985301Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985302Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985303Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985304Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985306Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985307Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1339985308Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2620850') ('4234', '20893122', '1339978692', None, None, None, None, None, '2620850') ('4234', None, None, None, None, None, None, None, '2620850') ('4234', '20893122', '1339978642', None, None, None, None, None, '2620850') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 272 time used for this insertion : 0.027865171432495117 save_final save missing photos in datou_result : time spend for datou_step_exec : 26.76598286628723 time spend to save output : 0.0316777229309082 total time spend for step 5 : 26.79766058921814 step6:thcl Thu Feb 27 21:21:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.0038900375366210938 time to convert the images to numpy array : 0.016999483108520508 time to import caffe and check if the image exist : 0.006206989288330078 time to convert the images to numpy array : 0.024831295013427734 time to import caffe and check if the image exist : 0.008854389190673828 time to convert the images to numpy array : 0.024719953536987305 time to import caffe and check if the image exist : 0.0068852901458740234 time to convert the images to numpy array : 0.028377532958984375 time to import caffe and check if the image exist : 0.00849008560180664 time to convert the images to numpy array : 0.02635788917541504 time to import caffe and check if the image exist : 0.016932249069213867 time to convert the images to numpy array : 0.019545793533325195 time to import caffe and check if the image exist : 0.011873722076416016 time to convert the images to numpy array : 0.025881528854370117 time to import caffe and check if the image exist : 0.009430170059204102 time to convert the images to numpy array : 0.028107643127441406 time to import caffe and check if the image exist : 0.008671760559082031 time to convert the images to numpy array : 0.02883148193359375 time to import caffe and check if the image exist : 0.01920318603515625 time to convert the images to numpy array : 0.019472837448120117 total time to convert the images to numpy array : 0.2831234931945801 list photo_ids error: [] list photo_ids correct : [1339985198, 1339985199, 1339985200, 1339985201, 1339985202, 1339985203, 1339985204, 1339985205, 1339985206, 1339985226, 1339985227, 1339985228, 1339985229, 1339985230, 1339985231, 1339985232, 1339985233, 1339985234, 1339985256, 1339985257, 1339985258, 1339985259, 1339985260, 1339985261, 1339985262, 1339985263, 1339985264, 1339985216, 1339985217, 1339985218, 1339985219, 1339985221, 1339985222, 1339985223, 1339985224, 1339985225, 1339985265, 1339985266, 1339985267, 1339985268, 1339985269, 1339985270, 1339985272, 1339985273, 1339985276, 1339985207, 1339985208, 1339985209, 1339985210, 1339985211, 1339985212, 1339985213, 1339985214, 1339985215, 1339985235, 1339985236, 1339985237, 1339985238, 1339985251, 1339985252, 1339985253, 1339985254, 1339985255, 1339985277, 1339985278, 1339985279, 1339985284, 1339985285, 1339985286, 1339985287, 1339985288, 1339985289, 1339985299, 1339985300, 1339985301, 1339985302, 1339985303, 1339985304, 1339985306, 1339985307, 1339985308, 1339985290, 1339985291, 1339985292, 1339985293, 1339985294, 1339985295, 1339985296, 1339985297, 1339985298] number of photos to traite : 90 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 15 wait 20 seconds l 3637 free memory gpu now : 15 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] WARNING: Logging before InitGoogleLogging() is written to STDERR E0227 21:21:50.931331 1269157 common.cpp:114] Cannot create Cublas handle. Cublas won't be available. E0227 21:21:50.936553 1269157 common.cpp:121] Cannot create Curand generator. Curand won't be available. local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory havn't enough memory gpu , need / 2500 l 3632 free memory gpu now : 15 wait 20 seconds F0227 21:22:12.746503 1269157 syncedmem.cpp:78] Check failed: error == cudaSuccess (2 vs. 0) out of memory *** Check failure stack trace: *** Command terminated by signal 6 27.59user 20.03system 1:48.32elapsed 43%CPU (0avgtext+0avgdata 2467176maxresident)k 92976inputs+4304outputs (11major+1736276minor)pagefaults 0swaps