python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 10 -a ' -a 4234 -l 5 ' -s datou_current_4234 -M 0 -S 0 -U 95,95,80 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3975779 load datou : 4234 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? output optionnel thcl was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load THCL from format json or kwargs add thcl : 3237 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig add pdt : 5561 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Current got : datou_id : 4234, datou_cur_ids : ['2601489'] with mtr_portfolio_ids : ['20695492'] and first list_photo_ids : [] new path : /proc/3975779/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, brightness, blur_detection, rle_unique_nms_with_priority, crop_condition, thcl, ventilate_hashtags_in_portfolio, final, velours_tree, send_mail_cod, split_time_score list_input_json : [] origin We have 1 , BFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 2 ; length of list_pids : 2 ; length of list_args : 2 time to download the photos : 0.3422975540161133 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 11 step1:mask_detect Thu Feb 20 23:20:28 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step mask_detect ! save_polygon : True begin detect begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 gpu_flag : 0 2025-02-20 23:20:31.656175: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-02-20 23:20:31.683132: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493065000 Hz 2025-02-20 23:20:31.685391: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fa8e8000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-02-20 23:20:31.685455: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-02-20 23:20:31.689690: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-02-20 23:20:31.920628: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3698b0c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-02-20 23:20:31.920676: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-02-20 23:20:31.922159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-20 23:20:31.926049: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-20 23:20:31.959375: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-20 23:20:31.962179: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-20 23:20:31.962919: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-20 23:20:31.966475: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-20 23:20:31.967886: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-20 23:20:31.972895: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-20 23:20:31.974322: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-20 23:20:31.974401: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-20 23:20:31.975159: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-20 23:20:31.975179: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-20 23:20:31.975188: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-20 23:20:31.976816: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) WARNING:tensorflow:From /home/admin/workarea/git/Velours/python/mtr/mask_rcnn/mask_detection.py:69: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead. 2025-02-20 23:20:32.306724: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-20 23:20:32.306847: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-20 23:20:32.306870: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-20 23:20:32.306891: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-20 23:20:32.306928: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-20 23:20:32.306948: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-20 23:20:32.306968: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-20 23:20:32.306989: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-20 23:20:32.308657: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-20 23:20:32.310064: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-02-20 23:20:32.310106: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-02-20 23:20:32.310129: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-20 23:20:32.310151: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-02-20 23:20:32.310172: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-02-20 23:20:32.310194: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-02-20 23:20:32.310215: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-02-20 23:20:32.310236: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-02-20 23:20:32.311942: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-02-20 23:20:32.311984: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-02-20 23:20:32.311996: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-02-20 23:20:32.312007: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-02-20 23:20:32.313328: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10023 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) Using TensorFlow backend. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:396: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:703: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. WARNING:tensorflow:From /home/admin/workarea/install/Mask_RCNN/model.py:729: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead. Inside mask_sub_process Inside mask_detect About to load cache.load_thcl_param To do loadFromThcl(), then load ParamDescType : thcl2847 thcls : [{'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}] thcl {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'} Update svm_hashtag_type_desc : 5275 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) {'thcl': {'id': 2847, 'mtr_user_id': 31, 'name': 'learn_RUBBIA_REFUS_AMIENS_23', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'photo_desc_type': 5275, 'type_classification': 'mask_rcnn', 'hashtag_id_list': '0,0,0,0,0,0,0,0,0'}, 'list_hashtags': ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'], 'list_hashtags_csv': 'background,papier,carton,metal,pet_clair,autre,pehd,pet_fonce,environnement', 'svm_portfolios_learning': '0,0,0,0,0,0,0,0,0', 'photo_hashtag_type': 3594, 'svm_hashtag_type_desc': 5275, 'photo_desc_type': 5275, 'pb_hashtag_id_or_classifier': 0} list_class_names : ['background', 'papier', 'carton', 'metal', 'pet_clair', 'autre', 'pehd', 'pet_fonce', 'environnement'] Configurations: BACKBONE resnet101 BACKBONE_SHAPES [[160 160] [ 80 80] [ 40 40] [ 20 20] [ 10 10]] BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.3 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 640 IMAGE_MIN_DIM 640 IMAGE_PADDING True IMAGE_SHAPE [640 640 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME learn_RUBBIA_REFUS_AMIENS_23 NUM_CLASSES 9 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (16, 32, 64, 128, 256) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001 model_param file didn't exist model_name : learn_RUBBIA_REFUS_AMIENS_23 model_type : mask_rcnn list file need : ['mask_model.h5'] file exist in s3 : ['mask_model.h5'] file manque in s3 : [] 2025-02-20 23:20:40.219888: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-02-20 23:20:40.527925: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 local folder : /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23 /data/models_weight/learn_RUBBIA_REFUS_AMIENS_23/mask_model.h5 size_local : 256009536 size in s3 : 256009536 create time local : 2021-08-09 09:43:22 create time in s3 : 2021-08-06 18:54:04 mask_model.h5 already exist and didn't need to update list_images length : 2 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -117.77031 max: 146.15859 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 9.465217590332031e-05 nb_pixel_total : 1132 time to create 1 rle with old method : 0.0020666122436523438 length of segment : 36 time for calcul the mask position with numpy : 6.246566772460938e-05 nb_pixel_total : 1055 time to create 1 rle with old method : 0.0019919872283935547 length of segment : 39 time for calcul the mask position with numpy : 8.034706115722656e-05 nb_pixel_total : 338 time to create 1 rle with old method : 0.0007021427154541016 length of segment : 62 time for calcul the mask position with numpy : 8.368492126464844e-05 nb_pixel_total : 1175 time to create 1 rle with old method : 0.002148866653442383 length of segment : 34 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.73516 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 5.078315734863281e-05 nb_pixel_total : 683 time to create 1 rle with old method : 0.0013747215270996094 length of segment : 34 time for calcul the mask position with numpy : 8.702278137207031e-05 nb_pixel_total : 2502 time to create 1 rle with old method : 0.0034072399139404297 length of segment : 51 time for calcul the mask position with numpy : 6.747245788574219e-05 nb_pixel_total : 615 time to create 1 rle with old method : 0.0008656978607177734 length of segment : 49 Processing 1 images image shape: (400, 400, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -105.50078 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0001251697540283203 nb_pixel_total : 5541 time to create 1 rle with old method : 0.007756471633911133 length of segment : 149 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -120.59063 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 6 time for calcul the mask position with numpy : 0.00020456314086914062 nb_pixel_total : 8399 time to create 1 rle with old method : 0.01097249984741211 length of segment : 119 time for calcul the mask position with numpy : 6.771087646484375e-05 nb_pixel_total : 761 time to create 1 rle with old method : 0.00104522705078125 length of segment : 51 time for calcul the mask position with numpy : 6.914138793945312e-05 nb_pixel_total : 998 time to create 1 rle with old method : 0.0012938976287841797 length of segment : 41 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 183 time to create 1 rle with old method : 0.0002791881561279297 length of segment : 16 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 1087 time to create 1 rle with old method : 0.0014753341674804688 length of segment : 42 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 248 time to create 1 rle with old method : 0.0004112720489501953 length of segment : 18 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 137.75625 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 4 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 530 time to create 1 rle with old method : 0.0008924007415771484 length of segment : 20 time for calcul the mask position with numpy : 7.82012939453125e-05 nb_pixel_total : 1411 time to create 1 rle with old method : 0.0017457008361816406 length of segment : 153 time for calcul the mask position with numpy : 0.0004825592041015625 nb_pixel_total : 40467 time to create 1 rle with old method : 0.050139665603637695 length of segment : 233 time for calcul the mask position with numpy : 0.0006289482116699219 nb_pixel_total : 47392 time to create 1 rle with old method : 0.054703474044799805 length of segment : 289 Processing 1 images image shape: (400, 400, 3) min: 26.00000 max: 213.00000 molded_images shape: (1, 640, 640, 3) min: -86.53984 max: 88.65078 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 4.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -108.20000 max: 151.02969 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 7.510185241699219e-05 nb_pixel_total : 2819 time to create 1 rle with old method : 0.0037169456481933594 length of segment : 46 time for calcul the mask position with numpy : 4.935264587402344e-05 nb_pixel_total : 320 time to create 1 rle with old method : 0.00048041343688964844 length of segment : 20 time for calcul the mask position with numpy : 5.745887756347656e-05 nb_pixel_total : 1208 time to create 1 rle with old method : 0.0016319751739501953 length of segment : 28 time for calcul the mask position with numpy : 5.4836273193359375e-05 nb_pixel_total : 305 time to create 1 rle with old method : 0.0004489421844482422 length of segment : 28 time for calcul the mask position with numpy : 0.0002048015594482422 nb_pixel_total : 9861 time to create 1 rle with old method : 0.012857437133789062 length of segment : 128 time for calcul the mask position with numpy : 6.29425048828125e-05 nb_pixel_total : 592 time to create 1 rle with old method : 0.0013179779052734375 length of segment : 19 time for calcul the mask position with numpy : 7.295608520507812e-05 nb_pixel_total : 1169 time to create 1 rle with old method : 0.0016198158264160156 length of segment : 63 time for calcul the mask position with numpy : 5.626678466796875e-05 nb_pixel_total : 1064 time to create 1 rle with old method : 0.0014998912811279297 length of segment : 40 time for calcul the mask position with numpy : 5.984306335449219e-05 nb_pixel_total : 228 time to create 1 rle with old method : 0.0003955364227294922 length of segment : 15 Processing 1 images image shape: (400, 400, 3) min: 7.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -104.21563 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 19 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 347 time to create 1 rle with old method : 0.0005135536193847656 length of segment : 19 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 786 time to create 1 rle with old method : 0.0010247230529785156 length of segment : 38 time for calcul the mask position with numpy : 9.250640869140625e-05 nb_pixel_total : 1720 time to create 1 rle with old method : 0.0022072792053222656 length of segment : 31 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 285 time to create 1 rle with old method : 0.00046515464782714844 length of segment : 26 time for calcul the mask position with numpy : 3.24249267578125e-05 nb_pixel_total : 145 time to create 1 rle with old method : 0.0002200603485107422 length of segment : 14 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 233 time to create 1 rle with old method : 0.00034928321838378906 length of segment : 14 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 118 time to create 1 rle with old method : 0.00021219253540039062 length of segment : 9 time for calcul the mask position with numpy : 0.00011587142944335938 nb_pixel_total : 4999 time to create 1 rle with old method : 0.005876779556274414 length of segment : 107 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 192 time to create 1 rle with old method : 0.0002636909484863281 length of segment : 45 time for calcul the mask position with numpy : 2.8848648071289062e-05 nb_pixel_total : 96 time to create 1 rle with old method : 0.00016570091247558594 length of segment : 12 time for calcul the mask position with numpy : 4.4345855712890625e-05 nb_pixel_total : 132 time to create 1 rle with old method : 0.0002868175506591797 length of segment : 16 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 249 time to create 1 rle with old method : 0.00037479400634765625 length of segment : 18 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 224 time to create 1 rle with old method : 0.00033473968505859375 length of segment : 15 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 1071 time to create 1 rle with old method : 0.00136566162109375 length of segment : 54 time for calcul the mask position with numpy : 5.7697296142578125e-05 nb_pixel_total : 1147 time to create 1 rle with old method : 0.0014798641204833984 length of segment : 44 time for calcul the mask position with numpy : 4.076957702636719e-05 nb_pixel_total : 617 time to create 1 rle with old method : 0.0008780956268310547 length of segment : 27 time for calcul the mask position with numpy : 3.170967102050781e-05 nb_pixel_total : 171 time to create 1 rle with old method : 0.0002684593200683594 length of segment : 15 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 235 time to create 1 rle with old method : 0.00036334991455078125 length of segment : 18 time for calcul the mask position with numpy : 9.5367431640625e-05 nb_pixel_total : 5208 time to create 1 rle with old method : 0.0060117244720458984 length of segment : 100 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -122.12578 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 3.838539123535156e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.000308990478515625 length of segment : 16 time for calcul the mask position with numpy : 0.0001842975616455078 nb_pixel_total : 5647 time to create 1 rle with old method : 0.007130146026611328 length of segment : 163 time for calcul the mask position with numpy : 0.00022983551025390625 nb_pixel_total : 7188 time to create 1 rle with old method : 0.008754968643188477 length of segment : 225 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.43594 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 4.124641418457031e-05 nb_pixel_total : 140 time to create 1 rle with old method : 0.00021791458129882812 length of segment : 29 time for calcul the mask position with numpy : 6.794929504394531e-05 nb_pixel_total : 1656 time to create 1 rle with old method : 0.0021326541900634766 length of segment : 78 time for calcul the mask position with numpy : 7.319450378417969e-05 nb_pixel_total : 1756 time to create 1 rle with old method : 0.002352476119995117 length of segment : 42 time for calcul the mask position with numpy : 0.00012826919555664062 nb_pixel_total : 811 time to create 1 rle with old method : 0.0020563602447509766 length of segment : 93 time for calcul the mask position with numpy : 6.270408630371094e-05 nb_pixel_total : 437 time to create 1 rle with old method : 0.0006544589996337891 length of segment : 24 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 558 time to create 1 rle with old method : 0.0008788108825683594 length of segment : 32 time for calcul the mask position with numpy : 0.00028634071350097656 nb_pixel_total : 4741 time to create 1 rle with old method : 0.00553584098815918 length of segment : 258 Processing 1 images image shape: (280, 400, 3) min: 17.00000 max: 217.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 99.62734 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0013153553009033203 nb_pixel_total : 105961 time to create 1 rle with old method : 0.17232346534729004 length of segment : 283 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.57656 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 17 time for calcul the mask position with numpy : 0.00010323524475097656 nb_pixel_total : 1931 time to create 1 rle with old method : 0.0033087730407714844 length of segment : 58 time for calcul the mask position with numpy : 0.00012350082397460938 nb_pixel_total : 3252 time to create 1 rle with old method : 0.005566596984863281 length of segment : 52 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 203 time to create 1 rle with old method : 0.0005183219909667969 length of segment : 10 time for calcul the mask position with numpy : 7.557868957519531e-05 nb_pixel_total : 800 time to create 1 rle with old method : 0.0016422271728515625 length of segment : 24 time for calcul the mask position with numpy : 5.316734313964844e-05 nb_pixel_total : 861 time to create 1 rle with old method : 0.0011646747589111328 length of segment : 42 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 111 time to create 1 rle with old method : 0.0001952648162841797 length of segment : 9 time for calcul the mask position with numpy : 3.981590270996094e-05 nb_pixel_total : 544 time to create 1 rle with old method : 0.0006978511810302734 length of segment : 36 time for calcul the mask position with numpy : 0.0001404285430908203 nb_pixel_total : 3138 time to create 1 rle with old method : 0.005465030670166016 length of segment : 106 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 465 time to create 1 rle with old method : 0.0006120204925537109 length of segment : 38 time for calcul the mask position with numpy : 8.392333984375e-05 nb_pixel_total : 1522 time to create 1 rle with old method : 0.0024793148040771484 length of segment : 49 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 209 time to create 1 rle with old method : 0.0002932548522949219 length of segment : 33 time for calcul the mask position with numpy : 0.00010371208190917969 nb_pixel_total : 2173 time to create 1 rle with old method : 0.002777576446533203 length of segment : 91 time for calcul the mask position with numpy : 0.00015234947204589844 nb_pixel_total : 5510 time to create 1 rle with old method : 0.006374835968017578 length of segment : 164 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 362 time to create 1 rle with old method : 0.0005471706390380859 length of segment : 23 time for calcul the mask position with numpy : 0.00010371208190917969 nb_pixel_total : 5924 time to create 1 rle with old method : 0.006996631622314453 length of segment : 155 time for calcul the mask position with numpy : 4.601478576660156e-05 nb_pixel_total : 394 time to create 1 rle with old method : 0.0005316734313964844 length of segment : 26 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 396 time to create 1 rle with old method : 0.0005702972412109375 length of segment : 27 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.85391 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 27 time for calcul the mask position with numpy : 6.699562072753906e-05 nb_pixel_total : 923 time to create 1 rle with old method : 0.0011532306671142578 length of segment : 40 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 639 time to create 1 rle with old method : 0.0008897781372070312 length of segment : 26 time for calcul the mask position with numpy : 4.649162292480469e-05 nb_pixel_total : 744 time to create 1 rle with old method : 0.0009813308715820312 length of segment : 38 time for calcul the mask position with numpy : 3.600120544433594e-05 nb_pixel_total : 273 time to create 1 rle with old method : 0.00040650367736816406 length of segment : 14 time for calcul the mask position with numpy : 3.361701965332031e-05 nb_pixel_total : 253 time to create 1 rle with old method : 0.0003654956817626953 length of segment : 14 time for calcul the mask position with numpy : 3.814697265625e-05 nb_pixel_total : 422 time to create 1 rle with old method : 0.0005970001220703125 length of segment : 20 time for calcul the mask position with numpy : 6.008148193359375e-05 nb_pixel_total : 1199 time to create 1 rle with old method : 0.001546621322631836 length of segment : 46 time for calcul the mask position with numpy : 6.461143493652344e-05 nb_pixel_total : 1779 time to create 1 rle with old method : 0.002231121063232422 length of segment : 51 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 1149 time to create 1 rle with old method : 0.0014946460723876953 length of segment : 48 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 866 time to create 1 rle with old method : 0.0011355876922607422 length of segment : 47 time for calcul the mask position with numpy : 6.031990051269531e-05 nb_pixel_total : 1618 time to create 1 rle with old method : 0.0020380020141601562 length of segment : 51 time for calcul the mask position with numpy : 6.604194641113281e-05 nb_pixel_total : 1382 time to create 1 rle with old method : 0.0021855831146240234 length of segment : 51 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 447 time to create 1 rle with old method : 0.0007920265197753906 length of segment : 23 time for calcul the mask position with numpy : 6.437301635742188e-05 nb_pixel_total : 1879 time to create 1 rle with old method : 0.0024514198303222656 length of segment : 49 time for calcul the mask position with numpy : 5.602836608886719e-05 nb_pixel_total : 1057 time to create 1 rle with old method : 0.0013048648834228516 length of segment : 45 time for calcul the mask position with numpy : 7.128715515136719e-05 nb_pixel_total : 1472 time to create 1 rle with old method : 0.0018773078918457031 length of segment : 75 time for calcul the mask position with numpy : 4.482269287109375e-05 nb_pixel_total : 411 time to create 1 rle with old method : 0.0005431175231933594 length of segment : 30 time for calcul the mask position with numpy : 4.2438507080078125e-05 nb_pixel_total : 633 time to create 1 rle with old method : 0.000835418701171875 length of segment : 29 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 364 time to create 1 rle with old method : 0.0004818439483642578 length of segment : 38 time for calcul the mask position with numpy : 6.246566772460938e-05 nb_pixel_total : 1700 time to create 1 rle with old method : 0.0021791458129882812 length of segment : 62 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 309 time to create 1 rle with old method : 0.0004260540008544922 length of segment : 35 time for calcul the mask position with numpy : 6.222724914550781e-05 nb_pixel_total : 1629 time to create 1 rle with old method : 0.0020322799682617188 length of segment : 60 time for calcul the mask position with numpy : 6.079673767089844e-05 nb_pixel_total : 321 time to create 1 rle with old method : 0.0006809234619140625 length of segment : 20 time for calcul the mask position with numpy : 0.00011801719665527344 nb_pixel_total : 1527 time to create 1 rle with old method : 0.0020444393157958984 length of segment : 64 time for calcul the mask position with numpy : 7.987022399902344e-05 nb_pixel_total : 1173 time to create 1 rle with old method : 0.0015370845794677734 length of segment : 60 time for calcul the mask position with numpy : 7.43865966796875e-05 nb_pixel_total : 1947 time to create 1 rle with old method : 0.002404451370239258 length of segment : 64 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 559 time to create 1 rle with old method : 0.0007619857788085938 length of segment : 30 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 146.62344 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 11 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 985 time to create 1 rle with old method : 0.0012691020965576172 length of segment : 29 time for calcul the mask position with numpy : 0.00033020973205566406 nb_pixel_total : 24857 time to create 1 rle with old method : 0.026415348052978516 length of segment : 276 time for calcul the mask position with numpy : 4.9591064453125e-05 nb_pixel_total : 289 time to create 1 rle with old method : 0.0004398822784423828 length of segment : 26 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 690 time to create 1 rle with old method : 0.0008938312530517578 length of segment : 42 time for calcul the mask position with numpy : 4.315376281738281e-05 nb_pixel_total : 593 time to create 1 rle with old method : 0.0008206367492675781 length of segment : 49 time for calcul the mask position with numpy : 4.220008850097656e-05 nb_pixel_total : 268 time to create 1 rle with old method : 0.00040531158447265625 length of segment : 26 time for calcul the mask position with numpy : 6.461143493652344e-05 nb_pixel_total : 457 time to create 1 rle with old method : 0.0006902217864990234 length of segment : 64 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 152 time to create 1 rle with old method : 0.00022792816162109375 length of segment : 20 time for calcul the mask position with numpy : 3.0517578125e-05 nb_pixel_total : 264 time to create 1 rle with old method : 0.0003733634948730469 length of segment : 27 time for calcul the mask position with numpy : 4.410743713378906e-05 nb_pixel_total : 107 time to create 1 rle with old method : 0.00016880035400390625 length of segment : 30 time for calcul the mask position with numpy : 3.337860107421875e-05 nb_pixel_total : 239 time to create 1 rle with old method : 0.0003609657287597656 length of segment : 21 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 254.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 137.35000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 182 time to create 1 rle with old method : 0.00034427642822265625 length of segment : 56 NEW PHOTO pour l'instant on ne peut pas sauvegarder la photo dans les tile Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -118.13359 max: 146.49062 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.3392181396484375e-05 nb_pixel_total : 1085 time to create 1 rle with old method : 0.0013172626495361328 length of segment : 32 time for calcul the mask position with numpy : 4.673004150390625e-05 nb_pixel_total : 1013 time to create 1 rle with old method : 0.0013761520385742188 length of segment : 36 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 1109 time to create 1 rle with old method : 0.001522064208984375 length of segment : 34 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -114.11406 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 883 time to create 1 rle with old method : 0.0011188983917236328 length of segment : 35 time for calcul the mask position with numpy : 6.341934204101562e-05 nb_pixel_total : 2454 time to create 1 rle with old method : 0.0030841827392578125 length of segment : 52 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 2757 time to create 1 rle with old method : 0.003278970718383789 length of segment : 55 Processing 1 images image shape: (400, 400, 3) min: 9.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -107.90703 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 3 time for calcul the mask position with numpy : 6.151199340820312e-05 nb_pixel_total : 305 time to create 1 rle with old method : 0.0006685256958007812 length of segment : 11 time for calcul the mask position with numpy : 9.226799011230469e-05 nb_pixel_total : 1428 time to create 1 rle with old method : 0.0017943382263183594 length of segment : 86 time for calcul the mask position with numpy : 9.489059448242188e-05 nb_pixel_total : 624 time to create 1 rle with old method : 0.0010478496551513672 length of segment : 79 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -120.90313 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.0001437664031982422 nb_pixel_total : 7896 time to create 1 rle with old method : 0.009261369705200195 length of segment : 133 time for calcul the mask position with numpy : 8.0108642578125e-05 nb_pixel_total : 778 time to create 1 rle with old method : 0.00090789794921875 length of segment : 62 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 253 time to create 1 rle with old method : 0.00037741661071777344 length of segment : 17 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 96 time to create 1 rle with old method : 0.00016379356384277344 length of segment : 16 time for calcul the mask position with numpy : 7.033348083496094e-05 nb_pixel_total : 2362 time to create 1 rle with old method : 0.00286102294921875 length of segment : 30 time for calcul the mask position with numpy : 4.506111145019531e-05 nb_pixel_total : 173 time to create 1 rle with old method : 0.0002696514129638672 length of segment : 16 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 1086 time to create 1 rle with old method : 0.0014035701751708984 length of segment : 42 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 138.18984 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.363059997558594e-05 nb_pixel_total : 545 time to create 1 rle with old method : 0.0007510185241699219 length of segment : 26 time for calcul the mask position with numpy : 2.9802322387695312e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.00025010108947753906 length of segment : 23 time for calcul the mask position with numpy : 5.841255187988281e-05 nb_pixel_total : 1331 time to create 1 rle with old method : 0.0016219615936279297 length of segment : 147 time for calcul the mask position with numpy : 3.790855407714844e-05 nb_pixel_total : 534 time to create 1 rle with old method : 0.0007486343383789062 length of segment : 20 time for calcul the mask position with numpy : 0.0004951953887939453 nb_pixel_total : 42398 time to create 1 rle with old method : 0.04655146598815918 length of segment : 289 Processing 1 images image shape: (400, 400, 3) min: 20.00000 max: 212.00000 molded_images shape: (1, 640, 640, 3) min: -87.03594 max: 90.96719 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 0 Processing 1 images image shape: (400, 400, 3) min: 11.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -99.32891 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 0.00017690658569335938 nb_pixel_total : 10045 time to create 1 rle with old method : 0.011507034301757812 length of segment : 126 time for calcul the mask position with numpy : 8.869171142578125e-05 nb_pixel_total : 2974 time to create 1 rle with old method : 0.0037851333618164062 length of segment : 48 time for calcul the mask position with numpy : 5.936622619628906e-05 nb_pixel_total : 934 time to create 1 rle with old method : 0.0013632774353027344 length of segment : 23 time for calcul the mask position with numpy : 3.695487976074219e-05 nb_pixel_total : 276 time to create 1 rle with old method : 0.00041413307189941406 length of segment : 23 time for calcul the mask position with numpy : 3.886222839355469e-05 nb_pixel_total : 533 time to create 1 rle with old method : 0.0008187294006347656 length of segment : 17 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 381 time to create 1 rle with old method : 0.0005640983581542969 length of segment : 23 time for calcul the mask position with numpy : 5.173683166503906e-05 nb_pixel_total : 1291 time to create 1 rle with old method : 0.0017151832580566406 length of segment : 40 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -115.87188 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 23 time for calcul the mask position with numpy : 6.985664367675781e-05 nb_pixel_total : 802 time to create 1 rle with old method : 0.001020193099975586 length of segment : 39 time for calcul the mask position with numpy : 3.314018249511719e-05 nb_pixel_total : 345 time to create 1 rle with old method : 0.0004515647888183594 length of segment : 17 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 105 time to create 1 rle with old method : 0.00019025802612304688 length of segment : 9 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 94 time to create 1 rle with old method : 0.00015664100646972656 length of segment : 12 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 186 time to create 1 rle with old method : 0.0003147125244140625 length of segment : 13 time for calcul the mask position with numpy : 3.2901763916015625e-05 nb_pixel_total : 377 time to create 1 rle with old method : 0.0005171298980712891 length of segment : 24 time for calcul the mask position with numpy : 3.0994415283203125e-05 nb_pixel_total : 267 time to create 1 rle with old method : 0.00035643577575683594 length of segment : 18 time for calcul the mask position with numpy : 4.887580871582031e-05 nb_pixel_total : 1247 time to create 1 rle with old method : 0.0015876293182373047 length of segment : 45 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 142 time to create 1 rle with old method : 0.0003402233123779297 length of segment : 11 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 217 time to create 1 rle with old method : 0.0004534721374511719 length of segment : 22 time for calcul the mask position with numpy : 7.534027099609375e-05 nb_pixel_total : 1172 time to create 1 rle with old method : 0.0021610260009765625 length of segment : 50 time for calcul the mask position with numpy : 4.458427429199219e-05 nb_pixel_total : 212 time to create 1 rle with old method : 0.0003147125244140625 length of segment : 38 time for calcul the mask position with numpy : 0.00017595291137695312 nb_pixel_total : 7511 time to create 1 rle with old method : 0.009157180786132812 length of segment : 128 time for calcul the mask position with numpy : 3.4809112548828125e-05 nb_pixel_total : 147 time to create 1 rle with old method : 0.00022673606872558594 length of segment : 17 time for calcul the mask position with numpy : 3.123283386230469e-05 nb_pixel_total : 130 time to create 1 rle with old method : 0.000209808349609375 length of segment : 16 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 262 time to create 1 rle with old method : 0.00038361549377441406 length of segment : 20 time for calcul the mask position with numpy : 4.5299530029296875e-05 nb_pixel_total : 844 time to create 1 rle with old method : 0.0010976791381835938 length of segment : 34 time for calcul the mask position with numpy : 4.0531158447265625e-05 nb_pixel_total : 334 time to create 1 rle with old method : 0.0005481243133544922 length of segment : 24 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 261 time to create 1 rle with old method : 0.0004127025604248047 length of segment : 27 time for calcul the mask position with numpy : 0.00011610984802246094 nb_pixel_total : 4823 time to create 1 rle with old method : 0.005956172943115234 length of segment : 114 time for calcul the mask position with numpy : 4.7206878662109375e-05 nb_pixel_total : 662 time to create 1 rle with old method : 0.0008590221405029297 length of segment : 64 time for calcul the mask position with numpy : 3.528594970703125e-05 nb_pixel_total : 234 time to create 1 rle with old method : 0.0003898143768310547 length of segment : 13 time for calcul the mask position with numpy : 3.075599670410156e-05 nb_pixel_total : 263 time to create 1 rle with old method : 0.0003502368927001953 length of segment : 20 Processing 1 images image shape: (400, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.14141 max: 151.10000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 5 time for calcul the mask position with numpy : 4.57763671875e-05 nb_pixel_total : 194 time to create 1 rle with old method : 0.0003018379211425781 length of segment : 17 time for calcul the mask position with numpy : 0.0001418590545654297 nb_pixel_total : 4918 time to create 1 rle with old method : 0.006191730499267578 length of segment : 200 time for calcul the mask position with numpy : 0.000247955322265625 nb_pixel_total : 5431 time to create 1 rle with old method : 0.006751060485839844 length of segment : 160 time for calcul the mask position with numpy : 0.00028204917907714844 nb_pixel_total : 7760 time to create 1 rle with old method : 0.00931549072265625 length of segment : 202 time for calcul the mask position with numpy : 0.0002918243408203125 nb_pixel_total : 8032 time to create 1 rle with old method : 0.009045124053955078 length of segment : 201 Processing 1 images image shape: (400, 320, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 144.43594 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 7 time for calcul the mask position with numpy : 7.867813110351562e-05 nb_pixel_total : 1693 time to create 1 rle with old method : 0.0020494461059570312 length of segment : 83 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 1008 time to create 1 rle with old method : 0.0015690326690673828 length of segment : 111 time for calcul the mask position with numpy : 5.412101745605469e-05 nb_pixel_total : 739 time to create 1 rle with old method : 0.0010564327239990234 length of segment : 57 time for calcul the mask position with numpy : 8.893013000488281e-05 nb_pixel_total : 1653 time to create 1 rle with old method : 0.002034425735473633 length of segment : 38 time for calcul the mask position with numpy : 0.0002636909484863281 nb_pixel_total : 3360 time to create 1 rle with old method : 0.004132747650146484 length of segment : 189 time for calcul the mask position with numpy : 8.726119995117188e-05 nb_pixel_total : 1690 time to create 1 rle with old method : 0.0020885467529296875 length of segment : 144 time for calcul the mask position with numpy : 0.0001862049102783203 nb_pixel_total : 3497 time to create 1 rle with old method : 0.004261493682861328 length of segment : 197 Processing 1 images image shape: (280, 400, 3) min: 19.00000 max: 217.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 93.60000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 0.0011856555938720703 nb_pixel_total : 106093 time to create 1 rle with old method : 0.117828369140625 length of segment : 282 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.68984 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 18 time for calcul the mask position with numpy : 7.82012939453125e-05 nb_pixel_total : 2283 time to create 1 rle with old method : 0.0029115676879882812 length of segment : 69 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 163 time to create 1 rle with old method : 0.0002722740173339844 length of segment : 10 time for calcul the mask position with numpy : 0.00011348724365234375 nb_pixel_total : 2537 time to create 1 rle with old method : 0.003515005111694336 length of segment : 98 time for calcul the mask position with numpy : 5.793571472167969e-05 nb_pixel_total : 761 time to create 1 rle with old method : 0.0009562969207763672 length of segment : 37 time for calcul the mask position with numpy : 7.605552673339844e-05 nb_pixel_total : 1083 time to create 1 rle with old method : 0.0015468597412109375 length of segment : 53 time for calcul the mask position with numpy : 7.724761962890625e-05 nb_pixel_total : 3210 time to create 1 rle with old method : 0.0038640499114990234 length of segment : 54 time for calcul the mask position with numpy : 4.100799560546875e-05 nb_pixel_total : 100 time to create 1 rle with old method : 0.00022673606872558594 length of segment : 10 time for calcul the mask position with numpy : 8.678436279296875e-05 nb_pixel_total : 3005 time to create 1 rle with old method : 0.003677845001220703 length of segment : 108 time for calcul the mask position with numpy : 8.726119995117188e-05 nb_pixel_total : 2419 time to create 1 rle with old method : 0.002838134765625 length of segment : 75 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 218 time to create 1 rle with old method : 0.00028967857360839844 length of segment : 31 time for calcul the mask position with numpy : 4.863739013671875e-05 nb_pixel_total : 805 time to create 1 rle with old method : 0.0010993480682373047 length of segment : 47 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 374 time to create 1 rle with old method : 0.0004971027374267578 length of segment : 24 time for calcul the mask position with numpy : 0.00011444091796875 nb_pixel_total : 3982 time to create 1 rle with old method : 0.004901409149169922 length of segment : 148 time for calcul the mask position with numpy : 6.866455078125e-05 nb_pixel_total : 506 time to create 1 rle with old method : 0.0006740093231201172 length of segment : 43 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 282 time to create 1 rle with old method : 0.0004985332489013672 length of segment : 44 time for calcul the mask position with numpy : 0.00012922286987304688 nb_pixel_total : 1015 time to create 1 rle with old method : 0.0021517276763916016 length of segment : 50 time for calcul the mask position with numpy : 5.507469177246094e-05 nb_pixel_total : 414 time to create 1 rle with old method : 0.000751495361328125 length of segment : 27 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 417 time to create 1 rle with old method : 0.0007555484771728516 length of segment : 27 Processing 1 images image shape: (280, 400, 3) min: 8.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 150.19766 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 28 time for calcul the mask position with numpy : 7.390975952148438e-05 nb_pixel_total : 1019 time to create 1 rle with old method : 0.0011875629425048828 length of segment : 47 time for calcul the mask position with numpy : 3.719329833984375e-05 nb_pixel_total : 306 time to create 1 rle with old method : 0.0004417896270751953 length of segment : 18 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 753 time to create 1 rle with old method : 0.0009350776672363281 length of segment : 41 time for calcul the mask position with numpy : 3.8623809814453125e-05 nb_pixel_total : 661 time to create 1 rle with old method : 0.0009143352508544922 length of segment : 26 time for calcul the mask position with numpy : 4.792213439941406e-05 nb_pixel_total : 1426 time to create 1 rle with old method : 0.0018320083618164062 length of segment : 42 time for calcul the mask position with numpy : 5.459785461425781e-05 nb_pixel_total : 270 time to create 1 rle with old method : 0.0006747245788574219 length of segment : 16 time for calcul the mask position with numpy : 4.553794860839844e-05 nb_pixel_total : 1356 time to create 1 rle with old method : 0.0016014575958251953 length of segment : 51 time for calcul the mask position with numpy : 6.794929504394531e-05 nb_pixel_total : 2002 time to create 1 rle with old method : 0.002480030059814453 length of segment : 67 time for calcul the mask position with numpy : 4.291534423828125e-05 nb_pixel_total : 624 time to create 1 rle with old method : 0.0008862018585205078 length of segment : 36 time for calcul the mask position with numpy : 3.3855438232421875e-05 nb_pixel_total : 438 time to create 1 rle with old method : 0.0005791187286376953 length of segment : 21 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 1762 time to create 1 rle with old method : 0.0022017955780029297 length of segment : 53 time for calcul the mask position with numpy : 4.8160552978515625e-05 nb_pixel_total : 1022 time to create 1 rle with old method : 0.0013244152069091797 length of segment : 50 time for calcul the mask position with numpy : 3.1948089599609375e-05 nb_pixel_total : 348 time to create 1 rle with old method : 0.0006041526794433594 length of segment : 21 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 1617 time to create 1 rle with old method : 0.0021827220916748047 length of segment : 51 time for calcul the mask position with numpy : 3.147125244140625e-05 nb_pixel_total : 127 time to create 1 rle with old method : 0.00023555755615234375 length of segment : 11 time for calcul the mask position with numpy : 6.651878356933594e-05 nb_pixel_total : 1975 time to create 1 rle with old method : 0.0025594234466552734 length of segment : 47 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 108 time to create 1 rle with old method : 0.00015544891357421875 length of segment : 21 time for calcul the mask position with numpy : 3.5762786865234375e-05 nb_pixel_total : 416 time to create 1 rle with old method : 0.0005373954772949219 length of segment : 22 time for calcul the mask position with numpy : 3.0040740966796875e-05 nb_pixel_total : 105 time to create 1 rle with old method : 0.00016236305236816406 length of segment : 16 time for calcul the mask position with numpy : 2.9087066650390625e-05 nb_pixel_total : 218 time to create 1 rle with old method : 0.0003006458282470703 length of segment : 14 time for calcul the mask position with numpy : 5.650520324707031e-05 nb_pixel_total : 1193 time to create 1 rle with old method : 0.0014352798461914062 length of segment : 56 time for calcul the mask position with numpy : 3.266334533691406e-05 nb_pixel_total : 301 time to create 1 rle with old method : 0.00044608116149902344 length of segment : 18 time for calcul the mask position with numpy : 3.743171691894531e-05 nb_pixel_total : 674 time to create 1 rle with old method : 0.0008625984191894531 length of segment : 26 time for calcul the mask position with numpy : 5.1975250244140625e-05 nb_pixel_total : 1309 time to create 1 rle with old method : 0.00164794921875 length of segment : 48 time for calcul the mask position with numpy : 3.504753112792969e-05 nb_pixel_total : 147 time to create 1 rle with old method : 0.0002560615539550781 length of segment : 20 time for calcul the mask position with numpy : 4.1484832763671875e-05 nb_pixel_total : 353 time to create 1 rle with old method : 0.0004684925079345703 length of segment : 38 time for calcul the mask position with numpy : 5.269050598144531e-05 nb_pixel_total : 1338 time to create 1 rle with old method : 0.0017311573028564453 length of segment : 57 time for calcul the mask position with numpy : 3.218650817871094e-05 nb_pixel_total : 204 time to create 1 rle with old method : 0.00030493736267089844 length of segment : 13 Processing 1 images image shape: (280, 400, 3) min: 0.00000 max: 255.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 147.30312 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 9 time for calcul the mask position with numpy : 3.7670135498046875e-05 nb_pixel_total : 202 time to create 1 rle with old method : 0.0003285408020019531 length of segment : 13 time for calcul the mask position with numpy : 3.62396240234375e-05 nb_pixel_total : 311 time to create 1 rle with old method : 0.00041413307189941406 length of segment : 36 time for calcul the mask position with numpy : 4.6253204345703125e-05 nb_pixel_total : 309 time to create 1 rle with old method : 0.0004990100860595703 length of segment : 25 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 745 time to create 1 rle with old method : 0.0010333061218261719 length of segment : 38 time for calcul the mask position with numpy : 4.839897155761719e-05 nb_pixel_total : 1051 time to create 1 rle with old method : 0.00139617919921875 length of segment : 31 time for calcul the mask position with numpy : 0.0002624988555908203 nb_pixel_total : 16457 time to create 1 rle with old method : 0.018950939178466797 length of segment : 245 time for calcul the mask position with numpy : 5.435943603515625e-05 nb_pixel_total : 1013 time to create 1 rle with old method : 0.0014798641204833984 length of segment : 71 time for calcul the mask position with numpy : 4.38690185546875e-05 nb_pixel_total : 531 time to create 1 rle with old method : 0.0007345676422119141 length of segment : 32 time for calcul the mask position with numpy : 4.9114227294921875e-05 nb_pixel_total : 127 time to create 1 rle with old method : 0.00024080276489257812 length of segment : 38 Processing 1 images image shape: (280, 320, 3) min: 0.00000 max: 253.00000 molded_images shape: (1, 640, 640, 3) min: -123.70000 max: 135.35000 image_metas shape: (1, 17) min: 0.00000 max: 640.00000 nb d'objets trouves : 1 time for calcul the mask position with numpy : 6.580352783203125e-05 nb_pixel_total : 191 time to create 1 rle with old method : 0.0004951953887939453 length of segment : 62 Detection mask done ! Trying to reset tf kernel 3976354 begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 2044 tf kernel not reseted sub process len(results) : 233 len(list_Values) 233 None max_time_sub_proc : 3600 parent process len(results) : 0 len(list_Values) 233 process is alive finish correctly or not : True after detect begin to check gpu status inside check gpu memory l 3610 free memory gpu now : 10814 Inside saveOutput : final : False verbose : 0 eke 12-6-18 : saveMask need to be cleaned for new output ! Catched exception ! Connect or reconnect ! Number saved : None batch 1 Loaded 233 chid ids of type : 4228 Number RLEs to save : 13117 save missing photos in datou_result : time spend for datou_step_exec : 23.67602849006653 time spend to save output : 1.4224894046783447 total time spend for step 1 : 25.098517894744873 step2:brightness Thu Feb 20 23:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness treat image : temp/1740090028_3975779_1338670666_41097d8e420bdd945a3e4ed902d43779.jpg treat image : temp/1740090028_3975779_1338670663_76778b9bbb21325071d7a34b9563ce9b.jpg Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.038628578186035156 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.015586137771606445 save missing photos in datou_result : time spend for datou_step_exec : 0.5037190914154053 time spend to save output : 0.05891132354736328 total time spend for step 2 : 0.5626304149627686 step3:blur_detection Thu Feb 20 23:20:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection methode: ratio et variance treat image : temp/1740090028_3975779_1338670666_41097d8e420bdd945a3e4ed902d43779.jpg resize: (1080, 1920) 1338670666 -7.726311783756201 treat image : temp/1740090028_3975779_1338670663_76778b9bbb21325071d7a34b9563ce9b.jpg resize: (1080, 1920) 1338670663 -7.676333768086701 Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 2 time used for this insertion : 0.016277790069580078 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 2 time used for this insertion : 0.014584541320800781 save missing photos in datou_result : time spend for datou_step_exec : 1.8006672859191895 time spend to save output : 0.037020206451416016 total time spend for step 3 : 1.8376874923706055 step4:rle_unique_nms_with_priority Thu Feb 20 23:20:56 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 233 chid ids of type : 4228 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++nb_obj : 111 nb_hashtags : 7 time to prepare the origin masks : 1.098606824874878 time for calcul the mask position with numpy : 0.05944561958312988 nb_pixel_total : 1770971 time to create 1 rle with new method : 0.16540765762329102 time for calcul the mask position with numpy : 0.008837223052978516 nb_pixel_total : 1411 time to create 1 rle with old method : 0.002416372299194336 time for calcul the mask position with numpy : 0.010205984115600586 nb_pixel_total : 40467 time to create 1 rle with old method : 0.04514122009277344 time for calcul the mask position with numpy : 0.008349180221557617 nb_pixel_total : 1132 time to create 1 rle with old method : 0.0012371540069580078 time for calcul the mask position with numpy : 0.00831747055053711 nb_pixel_total : 1055 time to create 1 rle with old method : 0.0012230873107910156 time for calcul the mask position with numpy : 0.009088754653930664 nb_pixel_total : 683 time to create 1 rle with old method : 0.0008540153503417969 time for calcul the mask position with numpy : 0.008231878280639648 nb_pixel_total : 114 time to create 1 rle with old method : 0.00018477439880371094 time for calcul the mask position with numpy : 0.0086669921875 nb_pixel_total : 2502 time to create 1 rle with old method : 0.0028429031372070312 time for calcul the mask position with numpy : 0.008637428283691406 nb_pixel_total : 11751 time to create 1 rle with old method : 0.013179779052734375 time for calcul the mask position with numpy : 0.010174036026000977 nb_pixel_total : 761 time to create 1 rle with old method : 0.0011048316955566406 time for calcul the mask position with numpy : 0.008816957473754883 nb_pixel_total : 615 time to create 1 rle with old method : 0.000759124755859375 time for calcul the mask position with numpy : 0.008514881134033203 nb_pixel_total : 248 time to create 1 rle with old method : 0.0002987384796142578 time for calcul the mask position with numpy : 0.008453845977783203 nb_pixel_total : 338 time to create 1 rle with old method : 0.0004105567932128906 time for calcul the mask position with numpy : 0.00854039192199707 nb_pixel_total : 183 time to create 1 rle with old method : 0.00022268295288085938 time for calcul the mask position with numpy : 0.008485555648803711 nb_pixel_total : 5541 time to create 1 rle with old method : 0.0067446231842041016 time for calcul the mask position with numpy : 0.008493185043334961 nb_pixel_total : 8399 time to create 1 rle with old method : 0.009185552597045898 time for calcul the mask position with numpy : 0.008350372314453125 nb_pixel_total : 96 time to create 1 rle with old method : 0.00015425682067871094 time for calcul the mask position with numpy : 0.008418560028076172 nb_pixel_total : 998 time to create 1 rle with old method : 0.0011506080627441406 time for calcul the mask position with numpy : 0.008458852767944336 nb_pixel_total : 530 time to create 1 rle with old method : 0.0006189346313476562 time for calcul the mask position with numpy : 0.008428335189819336 nb_pixel_total : 617 time to create 1 rle with old method : 0.0006954669952392578 time for calcul the mask position with numpy : 0.00834035873413086 nb_pixel_total : 171 time to create 1 rle with old method : 0.0002162456512451172 time for calcul the mask position with numpy : 0.008427143096923828 nb_pixel_total : 1988 time to create 1 rle with old method : 0.0025064945220947266 time for calcul the mask position with numpy : 0.008527517318725586 nb_pixel_total : 1720 time to create 1 rle with old method : 0.002028226852416992 time for calcul the mask position with numpy : 0.008871316909790039 nb_pixel_total : 5647 time to create 1 rle with old method : 0.006932258605957031 time for calcul the mask position with numpy : 0.008920907974243164 nb_pixel_total : 438 time to create 1 rle with old method : 0.0006070137023925781 time for calcul the mask position with numpy : 0.008487462997436523 nb_pixel_total : 4999 time to create 1 rle with old method : 0.005644083023071289 time for calcul the mask position with numpy : 0.008560657501220703 nb_pixel_total : 320 time to create 1 rle with old method : 0.0003883838653564453 time for calcul the mask position with numpy : 0.008710145950317383 nb_pixel_total : 1756 time to create 1 rle with old method : 0.0021626949310302734 time for calcul the mask position with numpy : 0.008472681045532227 nb_pixel_total : 140 time to create 1 rle with old method : 0.0001804828643798828 time for calcul the mask position with numpy : 0.008539676666259766 nb_pixel_total : 9861 time to create 1 rle with old method : 0.011096954345703125 time for calcul the mask position with numpy : 0.008518457412719727 nb_pixel_total : 194 time to create 1 rle with old method : 0.0003848075866699219 time for calcul the mask position with numpy : 0.009281396865844727 nb_pixel_total : 4741 time to create 1 rle with old method : 0.00583958625793457 time for calcul the mask position with numpy : 0.00856637954711914 nb_pixel_total : 558 time to create 1 rle with old method : 0.000713348388671875 time for calcul the mask position with numpy : 0.008559226989746094 nb_pixel_total : 331 time to create 1 rle with old method : 0.0004322528839111328 time for calcul the mask position with numpy : 0.008535385131835938 nb_pixel_total : 811 time to create 1 rle with old method : 0.001468658447265625 time for calcul the mask position with numpy : 0.008282661437988281 nb_pixel_total : 192 time to create 1 rle with old method : 0.00023365020751953125 time for calcul the mask position with numpy : 0.008284330368041992 nb_pixel_total : 1147 time to create 1 rle with old method : 0.0013458728790283203 time for calcul the mask position with numpy : 0.008377313613891602 nb_pixel_total : 118 time to create 1 rle with old method : 0.00015473365783691406 time for calcul the mask position with numpy : 0.008272886276245117 nb_pixel_total : 49 time to create 1 rle with old method : 0.000118255615234375 time for calcul the mask position with numpy : 0.008191823959350586 nb_pixel_total : 2819 time to create 1 rle with old method : 0.0032978057861328125 time for calcul the mask position with numpy : 0.008524894714355469 nb_pixel_total : 233 time to create 1 rle with old method : 0.0002956390380859375 time for calcul the mask position with numpy : 0.00856328010559082 nb_pixel_total : 786 time to create 1 rle with old method : 0.0011374950408935547 time for calcul the mask position with numpy : 0.008402347564697266 nb_pixel_total : 228 time to create 1 rle with old method : 0.0003020763397216797 time for calcul the mask position with numpy : 0.008297204971313477 nb_pixel_total : 249 time to create 1 rle with old method : 0.0003046989440917969 time for calcul the mask position with numpy : 0.008321046829223633 nb_pixel_total : 1656 time to create 1 rle with old method : 0.0019006729125976562 time for calcul the mask position with numpy : 0.008277654647827148 nb_pixel_total : 96 time to create 1 rle with old method : 0.00011420249938964844 time for calcul the mask position with numpy : 0.008575201034545898 nb_pixel_total : 132 time to create 1 rle with old method : 0.00017547607421875 time for calcul the mask position with numpy : 0.008185148239135742 nb_pixel_total : 305 time to create 1 rle with old method : 0.0003643035888671875 time for calcul the mask position with numpy : 0.008362531661987305 nb_pixel_total : 1071 time to create 1 rle with old method : 0.0012392997741699219 time for calcul the mask position with numpy : 0.008302450180053711 nb_pixel_total : 1064 time to create 1 rle with old method : 0.0012073516845703125 time for calcul the mask position with numpy : 0.008085966110229492 nb_pixel_total : 1169 time to create 1 rle with old method : 0.0013480186462402344 time for calcul the mask position with numpy : 0.008338689804077148 nb_pixel_total : 145 time to create 1 rle with old method : 0.00018215179443359375 time for calcul the mask position with numpy : 0.008500814437866211 nb_pixel_total : 224 time to create 1 rle with old method : 0.00028252601623535156 time for calcul the mask position with numpy : 0.008539676666259766 nb_pixel_total : 1208 time to create 1 rle with old method : 0.0014178752899169922 time for calcul the mask position with numpy : 0.008367538452148438 nb_pixel_total : 285 time to create 1 rle with old method : 0.0005381107330322266 time for calcul the mask position with numpy : 0.008469104766845703 nb_pixel_total : 347 time to create 1 rle with old method : 0.0004239082336425781 time for calcul the mask position with numpy : 0.008890867233276367 nb_pixel_total : 105961 time to create 1 rle with old method : 0.1402132511138916 time for calcul the mask position with numpy : 0.008354902267456055 nb_pixel_total : 203 time to create 1 rle with old method : 0.00025272369384765625 time for calcul the mask position with numpy : 0.008267879486083984 nb_pixel_total : 111 time to create 1 rle with old method : 0.00013947486877441406 time for calcul the mask position with numpy : 0.008274078369140625 nb_pixel_total : 447 time to create 1 rle with old method : 0.0005354881286621094 time for calcul the mask position with numpy : 0.008198738098144531 nb_pixel_total : 2173 time to create 1 rle with old method : 0.002519845962524414 time for calcul the mask position with numpy : 0.008169889450073242 nb_pixel_total : 3252 time to create 1 rle with old method : 0.0036678314208984375 time for calcul the mask position with numpy : 0.008366823196411133 nb_pixel_total : 690 time to create 1 rle with old method : 0.0008215904235839844 time for calcul the mask position with numpy : 0.008357048034667969 nb_pixel_total : 182 time to create 1 rle with old method : 0.00025081634521484375 time for calcul the mask position with numpy : 0.008036375045776367 nb_pixel_total : 209 time to create 1 rle with old method : 0.00023317337036132812 time for calcul the mask position with numpy : 0.007977724075317383 nb_pixel_total : 1527 time to create 1 rle with old method : 0.0016410350799560547 time for calcul the mask position with numpy : 0.008066892623901367 nb_pixel_total : 321 time to create 1 rle with old method : 0.00037670135498046875 time for calcul the mask position with numpy : 0.008007287979125977 nb_pixel_total : 1057 time to create 1 rle with old method : 0.0011632442474365234 time for calcul the mask position with numpy : 0.008204936981201172 nb_pixel_total : 253 time to create 1 rle with old method : 0.00028252601623535156 time for calcul the mask position with numpy : 0.008121013641357422 nb_pixel_total : 1173 time to create 1 rle with old method : 0.0013074874877929688 time for calcul the mask position with numpy : 0.008246183395385742 nb_pixel_total : 23872 time to create 1 rle with old method : 0.026282072067260742 time for calcul the mask position with numpy : 0.008134603500366211 nb_pixel_total : 615 time to create 1 rle with old method : 0.0007338523864746094 time for calcul the mask position with numpy : 0.007845163345336914 nb_pixel_total : 544 time to create 1 rle with old method : 0.000629425048828125 time for calcul the mask position with numpy : 0.00801229476928711 nb_pixel_total : 800 time to create 1 rle with old method : 0.0009086132049560547 time for calcul the mask position with numpy : 0.007981300354003906 nb_pixel_total : 5510 time to create 1 rle with old method : 0.0059719085693359375 time for calcul the mask position with numpy : 0.008249998092651367 nb_pixel_total : 422 time to create 1 rle with old method : 0.0004906654357910156 time for calcul the mask position with numpy : 0.008310317993164062 nb_pixel_total : 1629 time to create 1 rle with old method : 0.0019047260284423828 time for calcul the mask position with numpy : 0.008121728897094727 nb_pixel_total : 1931 time to create 1 rle with old method : 0.0021157264709472656 time for calcul the mask position with numpy : 0.008150577545166016 nb_pixel_total : 1779 time to create 1 rle with old method : 0.0019524097442626953 time for calcul the mask position with numpy : 0.00824427604675293 nb_pixel_total : 861 time to create 1 rle with old method : 0.0009307861328125 time for calcul the mask position with numpy : 0.008209228515625 nb_pixel_total : 639 time to create 1 rle with old method : 0.0006890296936035156 time for calcul the mask position with numpy : 0.008108139038085938 nb_pixel_total : 985 time to create 1 rle with old method : 0.0011396408081054688 time for calcul the mask position with numpy : 0.008011341094970703 nb_pixel_total : 1879 time to create 1 rle with old method : 0.0019686222076416016 time for calcul the mask position with numpy : 0.008266210556030273 nb_pixel_total : 107 time to create 1 rle with old method : 0.0001342296600341797 time for calcul the mask position with numpy : 0.00805521011352539 nb_pixel_total : 1149 time to create 1 rle with old method : 0.0013585090637207031 time for calcul the mask position with numpy : 0.008015632629394531 nb_pixel_total : 3138 time to create 1 rle with old method : 0.0034427642822265625 time for calcul the mask position with numpy : 0.008325338363647461 nb_pixel_total : 1451 time to create 1 rle with old method : 0.0016872882843017578 time for calcul the mask position with numpy : 0.008279085159301758 nb_pixel_total : 1700 time to create 1 rle with old method : 0.0019292831420898438 time for calcul the mask position with numpy : 0.008344411849975586 nb_pixel_total : 361 time to create 1 rle with old method : 0.0004105567932128906 time for calcul the mask position with numpy : 0.008244037628173828 nb_pixel_total : 866 time to create 1 rle with old method : 0.0009746551513671875 time for calcul the mask position with numpy : 0.008224964141845703 nb_pixel_total : 262 time to create 1 rle with old method : 0.00033020973205566406 time for calcul the mask position with numpy : 0.008215904235839844 nb_pixel_total : 1199 time to create 1 rle with old method : 0.0013468265533447266 time for calcul the mask position with numpy : 0.008352518081665039 nb_pixel_total : 3 time to create 1 rle with old method : 1.71661376953125e-05 time for calcul the mask position with numpy : 0.00814366340637207 nb_pixel_total : 36 time to create 1 rle with old method : 6.151199340820312e-05 time for calcul the mask position with numpy : 0.008201360702514648 nb_pixel_total : 57 time to create 1 rle with old method : 7.05718994140625e-05 time for calcul the mask position with numpy : 0.008090496063232422 nb_pixel_total : 268 time to create 1 rle with old method : 0.0002987384796142578 time for calcul the mask position with numpy : 0.008038759231567383 nb_pixel_total : 309 time to create 1 rle with old method : 0.0003643035888671875 time for calcul the mask position with numpy : 0.00830221176147461 nb_pixel_total : 457 time to create 1 rle with old method : 0.0005614757537841797 time for calcul the mask position with numpy : 0.00813150405883789 nb_pixel_total : 364 time to create 1 rle with old method : 0.0004744529724121094 time for calcul the mask position with numpy : 0.008054018020629883 nb_pixel_total : 465 time to create 1 rle with old method : 0.0005512237548828125 time for calcul the mask position with numpy : 0.008313417434692383 nb_pixel_total : 411 time to create 1 rle with old method : 0.0004830360412597656 time for calcul the mask position with numpy : 0.008241891860961914 nb_pixel_total : 1472 time to create 1 rle with old method : 0.001712799072265625 time for calcul the mask position with numpy : 0.008245706558227539 nb_pixel_total : 339 time to create 1 rle with old method : 0.0004413127899169922 time for calcul the mask position with numpy : 0.008114337921142578 nb_pixel_total : 1618 time to create 1 rle with old method : 0.0018463134765625 time for calcul the mask position with numpy : 0.008158445358276367 nb_pixel_total : 923 time to create 1 rle with old method : 0.0010216236114501953 time for calcul the mask position with numpy : 0.00814199447631836 nb_pixel_total : 89 time to create 1 rle with old method : 0.000110626220703125 time for calcul the mask position with numpy : 0.00813436508178711 nb_pixel_total : 744 time to create 1 rle with old method : 0.0008695125579833984 time for calcul the mask position with numpy : 0.008174419403076172 nb_pixel_total : 239 time to create 1 rle with old method : 0.00028586387634277344 time for calcul the mask position with numpy : 0.008147478103637695 nb_pixel_total : 13 time to create 1 rle with old method : 4.1961669921875e-05 time for calcul the mask position with numpy : 0.008370637893676758 nb_pixel_total : 633 time to create 1 rle with old method : 0.000701904296875 time for calcul the mask position with numpy : 0.008129119873046875 nb_pixel_total : 273 time to create 1 rle with old method : 0.00029349327087402344 time for calcul the mask position with numpy : 0.008140802383422852 nb_pixel_total : 19 time to create 1 rle with old method : 3.5762786865234375e-05 create new chi : 1.533684492111206 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.010959863662719727 batch 1 Loaded 112 chid ids of type : 4230 Number RLEs to save : 13028 TO DO : save crop sub photo not yet done ! save time : 1.0081515312194824 nb_obj : 119 nb_hashtags : 7 time to prepare the origin masks : 2.1203629970550537 time for calcul the mask position with numpy : 0.06711840629577637 nb_pixel_total : 1778761 time to create 1 rle with new method : 1.712000846862793 time for calcul the mask position with numpy : 0.009241819381713867 nb_pixel_total : 1428 time to create 1 rle with old method : 0.0016796588897705078 time for calcul the mask position with numpy : 0.007929086685180664 nb_pixel_total : 1331 time to create 1 rle with old method : 0.001478433609008789 time for calcul the mask position with numpy : 0.007348775863647461 nb_pixel_total : 226 time to create 1 rle with old method : 0.000324249267578125 time for calcul the mask position with numpy : 0.006842851638793945 nb_pixel_total : 42398 time to create 1 rle with old method : 0.04395914077758789 time for calcul the mask position with numpy : 0.0065021514892578125 nb_pixel_total : 883 time to create 1 rle with old method : 0.0009255409240722656 time for calcul the mask position with numpy : 0.006342411041259766 nb_pixel_total : 1013 time to create 1 rle with old method : 0.0011751651763916016 time for calcul the mask position with numpy : 0.006331205368041992 nb_pixel_total : 1085 time to create 1 rle with old method : 0.0011453628540039062 time for calcul the mask position with numpy : 0.0069043636322021484 nb_pixel_total : 74 time to create 1 rle with old method : 0.0001285076141357422 time for calcul the mask position with numpy : 0.006400346755981445 nb_pixel_total : 307 time to create 1 rle with old method : 0.00042510032653808594 time for calcul the mask position with numpy : 0.006586790084838867 nb_pixel_total : 2454 time to create 1 rle with old method : 0.0026924610137939453 time for calcul the mask position with numpy : 0.006398677825927734 nb_pixel_total : 778 time to create 1 rle with old method : 0.0008921623229980469 time for calcul the mask position with numpy : 0.0064716339111328125 nb_pixel_total : 138 time to create 1 rle with old method : 0.0002002716064453125 time for calcul the mask position with numpy : 0.006319761276245117 nb_pixel_total : 194 time to create 1 rle with old method : 0.0002238750457763672 time for calcul the mask position with numpy : 0.006505727767944336 nb_pixel_total : 173 time to create 1 rle with old method : 0.00021386146545410156 time for calcul the mask position with numpy : 0.006426334381103516 nb_pixel_total : 305 time to create 1 rle with old method : 0.0004048347473144531 time for calcul the mask position with numpy : 0.006398439407348633 nb_pixel_total : 7896 time to create 1 rle with old method : 0.00862431526184082 time for calcul the mask position with numpy : 0.00627446174621582 nb_pixel_total : 1086 time to create 1 rle with old method : 0.001241445541381836 time for calcul the mask position with numpy : 0.006245136260986328 nb_pixel_total : 183 time to create 1 rle with old method : 0.00024819374084472656 time for calcul the mask position with numpy : 0.006466865539550781 nb_pixel_total : 545 time to create 1 rle with old method : 0.0006561279296875 time for calcul the mask position with numpy : 0.006096601486206055 nb_pixel_total : 30 time to create 1 rle with old method : 6.413459777832031e-05 time for calcul the mask position with numpy : 0.006567478179931641 nb_pixel_total : 96 time to create 1 rle with old method : 0.0001385211944580078 time for calcul the mask position with numpy : 0.006569623947143555 nb_pixel_total : 662 time to create 1 rle with old method : 0.0007987022399902344 time for calcul the mask position with numpy : 0.0062220096588134766 nb_pixel_total : 2744 time to create 1 rle with old method : 0.003217935562133789 time for calcul the mask position with numpy : 0.006466865539550781 nb_pixel_total : 5431 time to create 1 rle with old method : 0.005842924118041992 time for calcul the mask position with numpy : 0.006121397018432617 nb_pixel_total : 4918 time to create 1 rle with old method : 0.005586385726928711 time for calcul the mask position with numpy : 0.0062389373779296875 nb_pixel_total : 334 time to create 1 rle with old method : 0.00044918060302734375 time for calcul the mask position with numpy : 0.006282329559326172 nb_pixel_total : 267 time to create 1 rle with old method : 0.000316619873046875 time for calcul the mask position with numpy : 0.006263017654418945 nb_pixel_total : 4823 time to create 1 rle with old method : 0.005308866500854492 time for calcul the mask position with numpy : 0.00614166259765625 nb_pixel_total : 186 time to create 1 rle with old method : 0.0002493858337402344 time for calcul the mask position with numpy : 0.006262063980102539 nb_pixel_total : 381 time to create 1 rle with old method : 0.000453948974609375 time for calcul the mask position with numpy : 0.0062329769134521484 nb_pixel_total : 1653 time to create 1 rle with old method : 0.001989603042602539 time for calcul the mask position with numpy : 0.006302595138549805 nb_pixel_total : 10045 time to create 1 rle with old method : 0.01124715805053711 time for calcul the mask position with numpy : 0.006563901901245117 nb_pixel_total : 7511 time to create 1 rle with old method : 0.008621454238891602 time for calcul the mask position with numpy : 0.006675004959106445 nb_pixel_total : 194 time to create 1 rle with old method : 0.00027871131896972656 time for calcul the mask position with numpy : 0.006486654281616211 nb_pixel_total : 584 time to create 1 rle with old method : 0.0007238388061523438 time for calcul the mask position with numpy : 0.006227016448974609 nb_pixel_total : 1247 time to create 1 rle with old method : 0.0014312267303466797 time for calcul the mask position with numpy : 0.006157398223876953 nb_pixel_total : 758 time to create 1 rle with old method : 0.0010128021240234375 time for calcul the mask position with numpy : 0.008034706115722656 nb_pixel_total : 1008 time to create 1 rle with old method : 0.0018396377563476562 time for calcul the mask position with numpy : 0.008614063262939453 nb_pixel_total : 396 time to create 1 rle with old method : 0.0005328655242919922 time for calcul the mask position with numpy : 0.008599996566772461 nb_pixel_total : 3360 time to create 1 rle with old method : 0.004002571105957031 time for calcul the mask position with numpy : 0.00746464729309082 nb_pixel_total : 1172 time to create 1 rle with old method : 0.0013668537139892578 time for calcul the mask position with numpy : 0.007196664810180664 nb_pixel_total : 8032 time to create 1 rle with old method : 0.008763551712036133 time for calcul the mask position with numpy : 0.007435798645019531 nb_pixel_total : 105 time to create 1 rle with old method : 0.00014328956604003906 time for calcul the mask position with numpy : 0.0073125362396240234 nb_pixel_total : 61 time to create 1 rle with old method : 0.0001513957977294922 time for calcul the mask position with numpy : 0.008304834365844727 nb_pixel_total : 2974 time to create 1 rle with old method : 0.0034036636352539062 time for calcul the mask position with numpy : 0.007776021957397461 nb_pixel_total : 142 time to create 1 rle with old method : 0.000194549560546875 time for calcul the mask position with numpy : 0.0075719356536865234 nb_pixel_total : 263 time to create 1 rle with old method : 0.00032901763916015625 time for calcul the mask position with numpy : 0.007098674774169922 nb_pixel_total : 261 time to create 1 rle with old method : 0.0003256797790527344 time for calcul the mask position with numpy : 0.00740361213684082 nb_pixel_total : 234 time to create 1 rle with old method : 0.0002999305725097656 time for calcul the mask position with numpy : 0.007814407348632812 nb_pixel_total : 802 time to create 1 rle with old method : 0.0009322166442871094 time for calcul the mask position with numpy : 0.007104158401489258 nb_pixel_total : 377 time to create 1 rle with old method : 0.0004382133483886719 time for calcul the mask position with numpy : 0.007658243179321289 nb_pixel_total : 1693 time to create 1 rle with old method : 0.0018076896667480469 time for calcul the mask position with numpy : 0.009418249130249023 nb_pixel_total : 262 time to create 1 rle with old method : 0.00032973289489746094 time for calcul the mask position with numpy : 0.00919342041015625 nb_pixel_total : 94 time to create 1 rle with old method : 0.00011801719665527344 time for calcul the mask position with numpy : 0.00945425033569336 nb_pixel_total : 147 time to create 1 rle with old method : 0.00017690658569335938 time for calcul the mask position with numpy : 0.009227275848388672 nb_pixel_total : 276 time to create 1 rle with old method : 0.0003342628479003906 time for calcul the mask position with numpy : 0.00951385498046875 nb_pixel_total : 1291 time to create 1 rle with old method : 0.0015690326690673828 time for calcul the mask position with numpy : 0.00919485092163086 nb_pixel_total : 130 time to create 1 rle with old method : 0.00017213821411132812 time for calcul the mask position with numpy : 0.009867191314697266 nb_pixel_total : 844 time to create 1 rle with old method : 0.0010271072387695312 time for calcul the mask position with numpy : 0.010334968566894531 nb_pixel_total : 189 time to create 1 rle with old method : 0.00029850006103515625 time for calcul the mask position with numpy : 0.010974884033203125 nb_pixel_total : 217 time to create 1 rle with old method : 0.0002541542053222656 time for calcul the mask position with numpy : 0.009592056274414062 nb_pixel_total : 934 time to create 1 rle with old method : 0.0011467933654785156 time for calcul the mask position with numpy : 0.009761571884155273 nb_pixel_total : 345 time to create 1 rle with old method : 0.0003631114959716797 time for calcul the mask position with numpy : 0.009867191314697266 nb_pixel_total : 100 time to create 1 rle with old method : 0.0001423358917236328 time for calcul the mask position with numpy : 0.010526657104492188 nb_pixel_total : 106093 time to create 1 rle with old method : 0.11247777938842773 time for calcul the mask position with numpy : 0.010465860366821289 nb_pixel_total : 163 time to create 1 rle with old method : 0.00021576881408691406 time for calcul the mask position with numpy : 0.009693622589111328 nb_pixel_total : 3210 time to create 1 rle with old method : 0.003744363784790039 time for calcul the mask position with numpy : 0.006782054901123047 nb_pixel_total : 2419 time to create 1 rle with old method : 0.002726316452026367 time for calcul the mask position with numpy : 0.006519794464111328 nb_pixel_total : 191 time to create 1 rle with old method : 0.0003070831298828125 time for calcul the mask position with numpy : 0.006365776062011719 nb_pixel_total : 218 time to create 1 rle with old method : 0.0002574920654296875 time for calcul the mask position with numpy : 0.006459712982177734 nb_pixel_total : 147 time to create 1 rle with old method : 0.00020933151245117188 time for calcul the mask position with numpy : 0.006451606750488281 nb_pixel_total : 147 time to create 1 rle with old method : 0.0001995563507080078 time for calcul the mask position with numpy : 0.006452798843383789 nb_pixel_total : 127 time to create 1 rle with old method : 0.0001533031463623047 time for calcul the mask position with numpy : 0.006673336029052734 nb_pixel_total : 348 time to create 1 rle with old method : 0.00043010711669921875 time for calcul the mask position with numpy : 0.00673675537109375 nb_pixel_total : 1022 time to create 1 rle with old method : 0.0012066364288330078 time for calcul the mask position with numpy : 0.00657200813293457 nb_pixel_total : 1193 time to create 1 rle with old method : 0.0013298988342285156 time for calcul the mask position with numpy : 0.006554603576660156 nb_pixel_total : 270 time to create 1 rle with old method : 0.0003437995910644531 time for calcul the mask position with numpy : 0.0069162845611572266 nb_pixel_total : 202 time to create 1 rle with old method : 0.00026154518127441406 time for calcul the mask position with numpy : 0.006826162338256836 nb_pixel_total : 1013 time to create 1 rle with old method : 0.0012164115905761719 time for calcul the mask position with numpy : 0.006618499755859375 nb_pixel_total : 3182 time to create 1 rle with old method : 0.0035195350646972656 time for calcul the mask position with numpy : 0.006785154342651367 nb_pixel_total : 306 time to create 1 rle with old method : 0.0003523826599121094 time for calcul the mask position with numpy : 0.006920814514160156 nb_pixel_total : 104 time to create 1 rle with old method : 0.00014257431030273438 time for calcul the mask position with numpy : 0.006706953048706055 nb_pixel_total : 1004 time to create 1 rle with old method : 0.001161336898803711 time for calcul the mask position with numpy : 0.006445884704589844 nb_pixel_total : 624 time to create 1 rle with old method : 0.0007603168487548828 time for calcul the mask position with numpy : 0.006474971771240234 nb_pixel_total : 2283 time to create 1 rle with old method : 0.002589702606201172 time for calcul the mask position with numpy : 0.006477832794189453 nb_pixel_total : 1762 time to create 1 rle with old method : 0.0019423961639404297 time for calcul the mask position with numpy : 0.0066792964935302734 nb_pixel_total : 15109 time to create 1 rle with old method : 0.016564607620239258 time for calcul the mask position with numpy : 0.009827375411987305 nb_pixel_total : 13 time to create 1 rle with old method : 6.580352783203125e-05 time for calcul the mask position with numpy : 0.0062634944915771484 nb_pixel_total : 2343 time to create 1 rle with old method : 0.0026557445526123047 time for calcul the mask position with numpy : 0.0063877105712890625 nb_pixel_total : 661 time to create 1 rle with old method : 0.0007870197296142578 time for calcul the mask position with numpy : 0.006194353103637695 nb_pixel_total : 22 time to create 1 rle with old method : 0.0001468658447265625 time for calcul the mask position with numpy : 0.006516456604003906 nb_pixel_total : 1051 time to create 1 rle with old method : 0.001188039779663086 time for calcul the mask position with numpy : 0.006975412368774414 nb_pixel_total : 557 time to create 1 rle with old method : 0.0006420612335205078 time for calcul the mask position with numpy : 0.0065691471099853516 nb_pixel_total : 75 time to create 1 rle with old method : 0.00010561943054199219 time for calcul the mask position with numpy : 0.00654149055480957 nb_pixel_total : 1975 time to create 1 rle with old method : 0.0021245479583740234 time for calcul the mask position with numpy : 0.006543159484863281 nb_pixel_total : 127 time to create 1 rle with old method : 0.00017261505126953125 time for calcul the mask position with numpy : 0.00635981559753418 nb_pixel_total : 805 time to create 1 rle with old method : 0.0008804798126220703 time for calcul the mask position with numpy : 0.006093263626098633 nb_pixel_total : 3003 time to create 1 rle with old method : 0.0034008026123046875 time for calcul the mask position with numpy : 0.00639653205871582 nb_pixel_total : 1426 time to create 1 rle with old method : 0.0015635490417480469 time for calcul the mask position with numpy : 0.006365537643432617 nb_pixel_total : 438 time to create 1 rle with old method : 0.0005271434783935547 time for calcul the mask position with numpy : 0.0062410831451416016 nb_pixel_total : 374 time to create 1 rle with old method : 0.0003936290740966797 time for calcul the mask position with numpy : 0.0065021514892578125 nb_pixel_total : 4 time to create 1 rle with old method : 4.0531158447265625e-05 time for calcul the mask position with numpy : 0.00627446174621582 nb_pixel_total : 43 time to create 1 rle with old method : 7.104873657226562e-05 time for calcul the mask position with numpy : 0.0064160823822021484 nb_pixel_total : 1356 time to create 1 rle with old method : 0.0015561580657958984 time for calcul the mask position with numpy : 0.006815910339355469 nb_pixel_total : 311 time to create 1 rle with old method : 0.0003745555877685547 time for calcul the mask position with numpy : 0.006562471389770508 nb_pixel_total : 353 time to create 1 rle with old method : 0.0004229545593261719 time for calcul the mask position with numpy : 0.006222724914550781 nb_pixel_total : 506 time to create 1 rle with old method : 0.0005884170532226562 time for calcul the mask position with numpy : 0.00616908073425293 nb_pixel_total : 2002 time to create 1 rle with old method : 0.002241373062133789 time for calcul the mask position with numpy : 0.006278038024902344 nb_pixel_total : 416 time to create 1 rle with old method : 0.00046896934509277344 time for calcul the mask position with numpy : 0.00598597526550293 nb_pixel_total : 1019 time to create 1 rle with old method : 0.001123189926147461 time for calcul the mask position with numpy : 0.00589442253112793 nb_pixel_total : 1083 time to create 1 rle with old method : 0.001329183578491211 time for calcul the mask position with numpy : 0.00807499885559082 nb_pixel_total : 36 time to create 1 rle with old method : 0.00011897087097167969 time for calcul the mask position with numpy : 0.008120298385620117 nb_pixel_total : 1309 time to create 1 rle with old method : 0.0013773441314697266 time for calcul the mask position with numpy : 0.007850408554077148 nb_pixel_total : 753 time to create 1 rle with old method : 0.000789642333984375 time for calcul the mask position with numpy : 0.008224010467529297 nb_pixel_total : 745 time to create 1 rle with old method : 0.0008687973022460938 time for calcul the mask position with numpy : 0.007921934127807617 nb_pixel_total : 309 time to create 1 rle with old method : 0.00036597251892089844 time for calcul the mask position with numpy : 0.007857084274291992 nb_pixel_total : 294 time to create 1 rle with old method : 0.0003333091735839844 time for calcul the mask position with numpy : 0.007900238037109375 nb_pixel_total : 218 time to create 1 rle with old method : 0.00025343894958496094 time for calcul the mask position with numpy : 0.007877826690673828 nb_pixel_total : 5 time to create 1 rle with old method : 1.4781951904296875e-05 create new chi : 2.974240303039551 after preparing all the mask , begin to delete the rle from the crop_hashtag_id => VR 28-11-20 : il faut déplacer cela apres le save_crop_hashtag_ids_obj de la ligne 9514 ! we have 0 chi objets contains the rles time to delete rle : 0.0015566349029541016 batch 1 Loaded 120 chid ids of type : 4230 Number RLEs to save : 13163 TO DO : save crop sub photo not yet done ! save time : 0.9840741157531738 map_output_result : {1338670666: (0.0, 'Should be the crop_list due to order', 0.0), 1338670663: (0.0, 'Should be the crop_list due to order', 0.0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1338670666, 1338670663] Looping around the photos to save general results len do output : 2 /1338670666.Didn't retrieve data . /1338670663.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670666', None, None, None, None, None, '2601489') ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670663', None, None, None, None, None, '2601489') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.014256954193115234 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.14624834060669 time spend to save output : 0.0144805908203125 total time spend for step 4 : 10.160728931427002 step5:crop_condition Thu Feb 20 23:21:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure some photos are not treated, begin crop_condition Loading chi in step crop with photo_hashtag_type : 4230 Loading chi in step crop for list_pids : 2 ! batch 1 Loaded 232 chid ids of type : 4230 begin to crop the class : papier param for this class : {'min_score': 0.6} filtre for class : papier hashtag_id of this class : 492668766 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 97 About to insert : list_path_to_insert length 97 new photo from crops ! About to upload 97 photos upload in portfolio : 4869462 init cache_photo without model_param we have 97 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740090068_3975779 we have uploaded 97 photos in the portfolio 4869462 time of upload the photos Elapsed time : 28.875452756881714 we have finished the crop for the class : papier begin to crop the class : carton param for this class : {'min_score': 0.6} filtre for class : carton hashtag_id of this class : 492774966 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 38 About to insert : list_path_to_insert length 38 new photo from crops ! About to upload 38 photos upload in portfolio : 4869462 init cache_photo without model_param we have 38 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740090098_3975779 we have uploaded 38 photos in the portfolio 4869462 time of upload the photos Elapsed time : 11.016284942626953 we have finished the crop for the class : carton begin to crop the class : metal param for this class : {'min_score': 0.6} filtre for class : metal hashtag_id of this class : 492628673 Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 2 About to insert : list_path_to_insert length 2 new photo from crops ! About to upload 2 photos upload in portfolio : 4869462 init cache_photo without model_param we have 2 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740090109_3975779 we have uploaded 2 photos in the portfolio 4869462 time of upload the photos Elapsed time : 0.8432669639587402 we have finished the crop for the class : metal begin to crop the class : pet_clair param for this class : {'min_score': 0.6} filtre for class : pet_clair hashtag_id of this class : 2107755846 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 10 About to insert : list_path_to_insert length 10 new photo from crops ! About to upload 10 photos upload in portfolio : 4869462 init cache_photo without model_param we have 10 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740090110_3975779 we have uploaded 10 photos in the portfolio 4869462 time of upload the photos Elapsed time : 3.2862801551818848 we have finished the crop for the class : pet_clair begin to crop the class : autre param for this class : {'min_score': 0.6} filtre for class : autre hashtag_id of this class : 494826614 Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! Next one ! map_result returned by crop_photo_return_map_crop : length : 16 About to insert : list_path_to_insert length 16 new photo from crops ! About to upload 16 photos upload in portfolio : 4869462 init cache_photo without model_param we have 16 photo to upload uploaded to storage server : ovh folder_temporaire : temp/1740090113_3975779 we have uploaded 16 photos in the portfolio 4869462 time of upload the photos Elapsed time : 5.201951742172241 we have finished the crop for the class : autre begin to crop the class : pehd param for this class : {'min_score': 0.6} filtre for class : pehd hashtag_id of this class : 628944319 begin to crop the class : pet_fonce param for this class : {'min_score': 0.6} filtre for class : pet_fonce hashtag_id of this class : 2107755900 delete rles for these photos Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : crop_condition we use saveGeneral [1338670666, 1338670663] Looping around the photos to save general results len do output : 163 /1338688982Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688983Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688984Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688985Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688986Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688987Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688988Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688989Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688990Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688991Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688992Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688993Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688994Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688995Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688996Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688997Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688998Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338688999Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689000Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689001Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689002Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689003Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689004Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689005Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689006Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689007Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689008Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689009Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689010Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689011Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689012Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689013Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689014Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689015Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689016Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689017Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689018Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689019Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689020Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689021Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689022Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689023Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689024Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689025Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689026Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689027Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689028Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689029Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689030Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689031Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689032Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689033Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689034Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689035Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689036Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689037Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689038Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689039Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689040Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689041Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689042Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689043Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689044Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689045Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689046Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689047Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689048Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689049Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689050Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689051Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689052Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689053Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689054Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689055Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689056Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689057Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689058Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689059Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689061Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689062Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689063Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689064Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689065Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689066Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689067Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689068Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689069Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689070Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689071Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689072Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689073Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689074Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689075Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689076Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689077Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689078Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689079Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689083Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689084Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689085Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689086Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689087Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689088Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689089Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689090Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689091Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689092Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689093Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689094Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689095Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689096Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689097Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689098Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689099Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689100Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689101Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689102Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689103Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689104Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689105Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689106Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689107Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689108Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689109Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689110Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689111Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689112Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689113Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689115Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689116Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689118Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689119Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689120Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689121Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689122Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689123Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689124Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689126Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689127Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689128Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689129Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689130Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689131Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689132Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689133Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689134Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689136Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689151Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689153Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689154Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689155Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689157Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689159Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689160Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689163Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689165Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689166Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689167Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689168Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689169Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689170Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689171Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . /1338689172Didn't retrieve data .Didn't retrieve data .Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670666', None, None, None, None, None, '2601489') ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670663', None, None, None, None, None, '2601489') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 491 time used for this insertion : 0.04568958282470703 save_final save missing photos in datou_result : time spend for datou_step_exec : 53.31758999824524 time spend to save output : 0.051886796951293945 total time spend for step 5 : 53.36947679519653 step6:thcl Thu Feb 20 23:21:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step Thcl ! we are using the classfication for only one thcl 3237 time to import caffe and check if the image exist : 0.004527568817138672 time to convert the images to numpy array : 0.033930063247680664 time to import caffe and check if the image exist : 0.007269620895385742 time to convert the images to numpy array : 0.043618202209472656 time to import caffe and check if the image exist : 0.018776893615722656 time to convert the images to numpy array : 0.0400850772857666 time to import caffe and check if the image exist : 0.011312007904052734 time to convert the images to numpy array : 0.049501895904541016 time to import caffe and check if the image exist : 0.018137454986572266 time to convert the images to numpy array : 0.04716992378234863 time to import caffe and check if the image exist : 0.015671730041503906 time to convert the images to numpy array : 0.050644636154174805 time to import caffe and check if the image exist : 0.018915653228759766 time to convert the images to numpy array : 0.05160260200500488 time to import caffe and check if the image exist : 0.02569723129272461 time to convert the images to numpy array : 0.04577803611755371 time to import caffe and check if the image exist : 0.021511077880859375 time to convert the images to numpy array : 0.051206350326538086 time to import caffe and check if the image exist : 0.015458822250366211 time to convert the images to numpy array : 0.058583974838256836 total time to convert the images to numpy array : 0.4227278232574463 list photo_ids error: [] list photo_ids correct : [1338689160, 1338689163, 1338689165, 1338689166, 1338689167, 1338689168, 1338689169, 1338689170, 1338689171, 1338689172, 1338689105, 1338689106, 1338689107, 1338689108, 1338689109, 1338689110, 1338689111, 1338689112, 1338689113, 1338689115, 1338689116, 1338689118, 1338689119, 1338689120, 1338689121, 1338689122, 1338689123, 1338689050, 1338689051, 1338689052, 1338689053, 1338689054, 1338689055, 1338689056, 1338689057, 1338689058, 1338689059, 1338689061, 1338689062, 1338689063, 1338689064, 1338689065, 1338689066, 1338689067, 1338689088, 1338689089, 1338689090, 1338689091, 1338689092, 1338689093, 1338689094, 1338689095, 1338689096, 1338689097, 1338689098, 1338689099, 1338689100, 1338689101, 1338689102, 1338689103, 1338689104, 1338688999, 1338689000, 1338689001, 1338689002, 1338689003, 1338689004, 1338689005, 1338689006, 1338689007, 1338689008, 1338689009, 1338689010, 1338689011, 1338689012, 1338689013, 1338689014, 1338689015, 1338689124, 1338689126, 1338689127, 1338689128, 1338689129, 1338689130, 1338689131, 1338689132, 1338689133, 1338689134, 1338689136, 1338689151, 1338689153, 1338689154, 1338689155, 1338689157, 1338689159, 1338688982, 1338688983, 1338688984, 1338688985, 1338688986, 1338688987, 1338688988, 1338688989, 1338688990, 1338688991, 1338688992, 1338688993, 1338688994, 1338688995, 1338688996, 1338688997, 1338688998, 1338689033, 1338689034, 1338689035, 1338689036, 1338689037, 1338689038, 1338689039, 1338689040, 1338689041, 1338689042, 1338689043, 1338689044, 1338689045, 1338689046, 1338689047, 1338689048, 1338689049, 1338689068, 1338689069, 1338689070, 1338689071, 1338689072, 1338689073, 1338689074, 1338689075, 1338689076, 1338689077, 1338689078, 1338689079, 1338689083, 1338689084, 1338689085, 1338689086, 1338689087, 1338689016, 1338689017, 1338689018, 1338689019, 1338689020, 1338689021, 1338689022, 1338689023, 1338689024, 1338689025, 1338689026, 1338689027, 1338689028, 1338689029, 1338689030, 1338689031, 1338689032] number of photos to traite : 163 try to delete the photos incorrect in DB tagging for thcl : 3237 To do loadFromThcl(), then load ParamDescType : thcl3237 thcls : [{'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'}] thcl {'id': 3237, 'mtr_user_id': 31, 'name': 'learn_rubbia_refus_2500', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Carton,Film_plastique,PEHD,PET_clair,PET_fonce,Papier,Tetrapak,flou,mal_croppe,metal,refus', 'svm_portfolios_learning': '4865689,4865690,4865686,4865684,4865685,4865688,4865691,4865693,4865692,4865687,4865683', 'photo_hashtag_type': 4158, 'photo_desc_type': 5561, 'type_classification': 'caffe', 'hashtag_id_list': '492774966,2107756122,628944319,2107755846,2107755900,492668766,609991870,492777938,2107755527,492628673,538914404'} Update svm_hashtag_type_desc : 5561 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) To loadFromThcl() : net_5561 begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10814 max_wait_temp : 1 max_wait : 0 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5561, 'learn_rubbia_refus_2500', 2048, 2048, 'learn_rubbia_refus_2500', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2021, 12, 2, 19, 10, 8), datetime.datetime(2021, 12, 2, 19, 10, 8)) None mean_file_type : mean_file_path : prototxt_file_path : model : learn_rubbia_refus_2500 Inside get_net Inside get_net before cache_data_model model_param file didn't exist Inside get_net before CDM.load_model_par_type model_name : learn_rubbia_refus_2500 model_type : caffe list file need : ['caffemodel', 'deploy_conv_normal.prototxt', 'deploy_fc.prototxt', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file exist in s3 : ['caffemodel', 'deploy.prototxt', 'mean.npy', 'synset_words.txt'] file manque in s3 : ['deploy_conv_normal.prototxt', 'deploy_fc.prototxt'] local folder : /data/models_weight/learn_rubbia_refus_2500 /data/models_weight/learn_rubbia_refus_2500/caffemodel size_local : 94358479 size in s3 : 94358479 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:16 caffemodel already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt size_local : 32544 size in s3 : 32544 create time local : 2021-12-03 18:29:39 create time in s3 : 2021-12-02 17:49:15 deploy.prototxt already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/mean.npy size_local : 1572992 size in s3 : 1572992 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:09:52 mean.npy already exist and didn't need to update /data/models_weight/learn_rubbia_refus_2500/synset_words.txt size_local : 334 size in s3 : 334 create time local : 2021-12-03 18:29:40 create time in s3 : 2021-12-02 18:10:06 synset_words.txt already exist and didn't need to update Inside get_net after CDM.load_model_par_type After if not only_with_local_cache: /home/admin/workarea/install/caffe_cuda8_python3/python/:/home/admin/workarea/install/darknet/:/home/admin/workarea/git/Velours/python:/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python:/home/admin/mtr/.credentials:/home/admin/workarea/install/caffe/python:/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools/:/home/admin/workarea/git/fotonowerpip/:/home/admin/workarea/install/segment-anything:/home/admin//workarea/git/pyfvs/ Here before set mode gpu Doing nothing but we could set mode gpu after set mode gpu prototxt_filename : /data/models_weight/learn_rubbia_refus_2500/deploy.prototxt caffemodel_filename : /data/models_weight/learn_rubbia_refus_2500/caffemodel now we set caffe to gpu mode before predict begin to check gpu status inside check gpu memory l 3637 free memory gpu now : 10595 max_wait_temp : 1 max_wait : 0 dict_keys(['prob', 'pool5']) time used to do the prepocess of the images : 0.7673571109771729 time used to do the prediction : 0.697882890701294 save descriptor for thcl : 3237 time to traite the descriptors : 0.9079492092132568 Catched exception ! Connect or reconnect ! storage_type for insertDescriptorsMulti : 3 To insert : 1338689160 To insert : 1338689163 To insert : 1338689165 To insert : 1338689166 To insert : 1338689167 To insert : 1338689168 To insert : 1338689169 To insert : 1338689170 To insert : 1338689171 To insert : 1338689172 To insert : 1338689105 To insert : 1338689106 To insert : 1338689107 To insert : 1338689108 To insert : 1338689109 To insert : 1338689110 To insert : 1338689111 To insert : 1338689112 To insert : 1338689113 To insert : 1338689115 To insert : 1338689116 To insert : 1338689118 To insert : 1338689119 To insert : 1338689120 To insert : 1338689121 To insert : 1338689122 To insert : 1338689123 To insert : 1338689050 To insert : 1338689051 To insert : 1338689052 To insert : 1338689053 To insert : 1338689054 To insert : 1338689055 To insert : 1338689056 To insert : 1338689057 To insert : 1338689058 To insert : 1338689059 To insert : 1338689061 To insert : 1338689062 To insert : 1338689063 To insert : 1338689064 To insert : 1338689065 To insert : 1338689066 To insert : 1338689067 To insert : 1338689088 To insert : 1338689089 To insert : 1338689090 To insert : 1338689091 To insert : 1338689092 To insert : 1338689093 To insert : 1338689094 To insert : 1338689095 To insert : 1338689096 To insert : 1338689097 To insert : 1338689098 To insert : 1338689099 To insert : 1338689100 To insert : 1338689101 To insert : 1338689102 To insert : 1338689103 To insert : 1338689104 To insert : 1338688999 To insert : 1338689000 To insert : 1338689001 To insert : 1338689002 To insert : 1338689003 To insert : 1338689004 To insert : 1338689005 To insert : 1338689006 To insert : 1338689007 To insert : 1338689008 To insert : 1338689009 To insert : 1338689010 To insert : 1338689011 To insert : 1338689012 To insert : 1338689013 To insert : 1338689014 To insert : 1338689015 To insert : 1338689124 To insert : 1338689126 To insert : 1338689127 To insert : 1338689128 To insert : 1338689129 To insert : 1338689130 To insert : 1338689131 To insert : 1338689132 To insert : 1338689133 To insert : 1338689134 To insert : 1338689136 To insert : 1338689151 To insert : 1338689153 To insert : 1338689154 To insert : 1338689155 To insert : 1338689157 To insert : 1338689159 To insert : 1338688982 To insert : 1338688983 To insert : 1338688984 To insert : 1338688985 To insert : 1338688986 To insert : 1338688987 To insert : 1338688988 To insert : 1338688989 To insert : 1338688990 To insert : 1338688991 To insert : 1338688992 To insert : 1338688993 To insert : 1338688994 To insert : 1338688995 To insert : 1338688996 To insert : 1338688997 To insert : 1338688998 To insert : 1338689033 To insert : 1338689034 To insert : 1338689035 To insert : 1338689036 To insert : 1338689037 To insert : 1338689038 To insert : 1338689039 To insert : 1338689040 To insert : 1338689041 To insert : 1338689042 To insert : 1338689043 To insert : 1338689044 To insert : 1338689045 To insert : 1338689046 To insert : 1338689047 To insert : 1338689048 To insert : 1338689049 To insert : 1338689068 To insert : 1338689069 To insert : 1338689070 To insert : 1338689071 To insert : 1338689072 To insert : 1338689073 To insert : 1338689074 To insert : 1338689075 To insert : 1338689076 To insert : 1338689077 To insert : 1338689078 To insert : 1338689079 To insert : 1338689083 To insert : 1338689084 To insert : 1338689085 To insert : 1338689086 To insert : 1338689087 To insert : 1338689016 To insert : 1338689017 To insert : 1338689018 To insert : 1338689019 To insert : 1338689020 To insert : 1338689021 To insert : 1338689022 To insert : 1338689023 To insert : 1338689024 To insert : 1338689025 To insert : 1338689026 To insert : 1338689027 To insert : 1338689028 To insert : 1338689029 To insert : 1338689030 To insert : 1338689031 To insert : 1338689032 time to insert the descriptors : 39.31764078140259 Inside saveOutput : final : False verbose : 0 time used to find the portfolios of the photos SAVE THCL : begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 247 time used for this insertion : 0.02705097198486328 save missing photos in datou_result : time spend for datou_step_exec : 46.036722898483276 time spend to save output : 0.07900834083557129 total time spend for step 6 : 46.11573123931885 step7:ventilate_hashtags_in_portfolio Thu Feb 20 23:22:46 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 20695492 get user id for portfolio 20695492 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20695492 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton','Film_plastique','environnement','carton','PET_fonce','flou','pet_clair','mal_croppe','PET_clair','Tetrapak','metal','PEHD','papier','autre','pet_fonce','refus','pehd','Papier')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20695492 AND mptpi.`type`=4230 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton','Film_plastique','environnement','carton','PET_fonce','flou','pet_clair','mal_croppe','PET_clair','Tetrapak','metal','PEHD','papier','autre','pet_fonce','refus','pehd','Papier')) AND mptpi.`min_score`=0.6 To do Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") Catched exception ! Connect or reconnect ! (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')\n and cspi.crop_hashtag_id = chi.id' at line 3") To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20695492 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton','Film_plastique','environnement','carton','PET_fonce','flou','pet_clair','mal_croppe','PET_clair','Tetrapak','metal','PEHD','papier','autre','pet_fonce','refus','pehd','Papier')) AND mptpi.`min_score`=0.6 To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20695492 AND mptpi.`type`=4231 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('Carton','Film_plastique','environnement','carton','PET_fonce','flou','pet_clair','mal_croppe','PET_clair','Tetrapak','metal','PEHD','papier','autre','pet_fonce','refus','pehd','Papier')) AND mptpi.`min_score`=0.6 To do lien utilise dans velours : https://www.fotonower.com/velours/20697549,20697547,20697548,20697560,20697551,20697554,20697553,20697555,20697556,20697562,20697563,20697559,20697561?tags=carton,film_plastique,environnement,pet_fonce,flou,pet_clair,mal_croppe,tetrapak,metal,pehd,papier,autre,refus&datou_id_consolidate=4235&port_consolidate=20695492 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1338670666, 1338670663] Looping around the photos to save general results len do output : 1 /20695492. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670666', None, None, None, None, None, '2601489') ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670663', None, None, None, None, None, '2601489') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.013041257858276367 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.539891481399536 time spend to save output : 0.013335227966308594 total time spend for step 7 : 2.5532267093658447 step8:final Thu Feb 20 23:22:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Catched exception ! Connect or reconnect ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1338670666: ('0.24039617079408773',), 1338670663: ('0.24039617079408773',)} new output for save of step final : {1338670666: ('0.24039617079408773',), 1338670663: ('0.24039617079408773',)} [1338670666, 1338670663] Looping around the photos to save general results len do output : 2 /1338670666.Didn't retrieve data . /1338670663.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670666', None, None, None, None, None, '2601489') ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670663', None, None, None, None, None, '2601489') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 6 time used for this insertion : 0.01145792007446289 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.12034893035888672 time spend to save output : 0.011658191680908203 total time spend for step 8 : 0.13200712203979492 step9:velours_tree Thu Feb 20 23:22:48 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 VR 22-3-18 : For now we do not clean correctly the datou structure list_portfolios : 20697549,20697547,20697548,20697560,20697551,20697554,20697553,20697555,20697556,20697562,20697563,20697559,20697561 photo desc type : 5561 - Retrieving photos to tag... query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697549 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697547 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697548 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697560 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697551 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697554 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697553 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697555 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697556 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697562 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697563 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697559 ORDER BY ph.size desc query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20697561 ORDER BY ph.size desc - Loading descriptors... Size : 2048 len(descriptors) : 163 Compute structured hierarchical clustering... ward : AgglomerativeClustering(n_clusters=6) ward.labels_ : [5 3 4 0 5 0 5 1 2 1 1 2 0 1 1 0 0 0 5 4 5 5 0 0 1 1 1 2 1 1 1 2 0 1 5 1 0 1 4 1 0 0 2 4 5 4 3 4 0 5 0 5 0 1 2 1 0 0 1 0 1 0 5 1 0 0 1 0 4 5 0 0 0 2 1 1 1 1 1 1 2 1 2 1 1 1 0 4 2 1 0 1 2 2 2 4 4 3 3 0 2 1 2 1 2 2 2 2 1 2 4 4 2 3 3 0 5 2 1 2 2 1 2 2 5 2 1 1 4 4 2 2 1 1 5 5 2 1 1 4 1 0 1 5 1 5 5 4 5 5 0 4 5 5 5 2 1 5 4 1 5 5 5] Elapsed time: 0.021262645721435547 graph_id used : 79508 - Beta version, working pretty good on 11-5-16 ! https://www.fotonower.com/velours/20697549,20697547,20697548,20697560,20697551,20697554,20697553,20697555,20697556,20697562,20697563,20697559,20697561?tags=carton,film_plastique,environnement,pet_fonce,flou,pet_clair,mal_croppe,tetrapak,metal,pehd,papier,autre,refus&datou_id_consolidate=4235&port_consolidate=20695492&tree_id=79508 Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : velours_tree we use saveGeneral [1338670666, 1338670663] Looping around the photos to save general results len do output : 1 /20695492Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670666', None, None, None, None, None, '2601489') ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670663', None, None, None, None, None, '2601489') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.01603531837463379 save_final save missing photos in datou_result : time spend for datou_step_exec : 10.65874695777893 time spend to save output : 0.016254186630249023 total time spend for step 9 : 10.67500114440918 step10:send_mail_cod Thu Feb 20 23:22:59 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 2 complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_Qualipapia_P20695492_20-02-2025_23_22_59.pdf 20697530 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206975301740090179 20697528 imagette206975281740090180 20697541 imagette206975411740090180 20697532 imagette206975321740090180 20697535 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206975351740090180 20697534 imagette206975341740090180 20697536 imagette206975361740090180 20697537 change filename to text .change filename to text .imagette206975371740090180 20697543 imagette206975431740090180 20697544 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206975441740090180 20697540 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette206975401740090181 20697542 imagette206975421740090182 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=20695492 and hashtag_type = 4230 and pcr.hashtag_id = h.hashtag_id; velour_link : https://www.fotonower.com/velours/20697549,20697547,20697548,20697560,20697551,20697554,20697553,20697555,20697556,20697562,20697563,20697559,20697561?tags=carton,film_plastique,environnement,pet_fonce,flou,pet_clair,mal_croppe,tetrapak,metal,pehd,papier,autre,refus&datou_id_consolidate=4235&port_consolidate=20695492&tree_id=79508 your option no_mail is active, we will not send the real mail to your client args[1338670666] : ((1338670666, -7.726311783756201, 492609224), (1338670666, -0.20728661597390094, 496442774), '0.24039617079408773') We are sending mail with results at cod@fotonower.com args[1338670663] : ((1338670663, -7.676333768086701, 492609224), (1338670663, -0.20445095941750696, 496442774), '0.24039617079408773') We are sending mail with results at cod@fotonower.com refus_total : 0.24039617079408773 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=20695492 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20695492_20-02-2025_23_22_59.pdf results_Qualipapia_P20695492_20-02-2025_23_22_59.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20695492_20-02-2025_23_22_59.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4234','20695492','results_Qualipapia_P20695492_20-02-2025_23_22_59.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20695492_20-02-2025_23_22_59.pdf','pdf','','0.14','0.24039617079408773') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1338670666, 1338670663] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670666', None, None, None, None, None, '2601489') ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670663', None, None, None, None, None, '2601489') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 2 time used for this insertion : 0.012579202651977539 save_final save missing photos in datou_result : time spend for datou_step_exec : 3.9054181575775146 time spend to save output : 0.012734174728393555 total time spend for step 10 : 3.918152332305908 step11:split_time_score Thu Feb 20 23:23:03 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score Catched exception ! Connect or reconnect ! TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3442, 'mtr_user_id': 31, 'name': 'classifieur_2camions_valcor_021122_v1', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'deux_camions,camion_droite,camion_gauche,pas_de_camion', 'svm_portfolios_learning': '7659379,7659034,7657685,7657114', 'photo_hashtag_type': 4458, 'photo_desc_type': 5723, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107760533,2107760534,2107760535,2107760536'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('13', 2),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 20022025 20695492 Nombre de photos uploadées : 2 / 23040 (0%) 20022025 20695492 Nombre de photos taguées (types de déchets): 2 / 2 (100%) 20022025 20695492 Nombre de photos taguées (volume) : 0 / 2 (0%) elapsed_time : load_data_split_time_score 1.9073486328125e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ?? elapsed_time : fill_and_build_computed_from_old_data 0.000179290771484375 elapsed_time : insert_dashboard_record_day_entry 0.02871990203857422 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps Qualite : 0.22084362600633364 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20686959_20-02-2025_19_22_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20686959 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20686959 AND mptpi.`type`=4230 To do Qualite : 0.2796462772587954 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20688282_20-02-2025_19_48_51.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20688282 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20688282 AND mptpi.`type`=4230 To do find url: select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20695489 order by id desc limit 1 Qualite : 0.2883970161379565 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20695490_20-02-2025_22_49_52.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20695490 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20695490 AND mptpi.`type`=4230 To do Qualite : 0.24039617079408773 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Qualipapia_P20695492_20-02-2025_23_22_59.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 20695492 order by id desc limit 1 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20695492 AND mptpi.`type`=4230 To do NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'20022025': {'nb_upload': 2, 'nb_taggue_class': 2, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1338670666, 1338670663] Looping around the photos to save general results len do output : 1 /20695492Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670666', None, None, None, None, None, '2601489') ('4234', None, None, None, None, None, None, None, '2601489') ('4234', '20695492', '1338670663', None, None, None, None, None, '2601489') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 3 time used for this insertion : 0.012720584869384766 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.339728832244873 time spend to save output : 0.01295328140258789 total time spend for step 11 : 1.352682113647461 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 1 len(input) + len(total_photo_id_missing) : 2 set_done_treatment 33.43user 24.37system 2:39.96elapsed 36%CPU (0avgtext+0avgdata 2505300maxresident)k 1031864inputs+12216outputs (1874major+2128784minor)pagefaults 0swaps